The time course of multimodal percepts induced by reliability changes
Single Paper Presentation

Massimiliano Di Luca
Max Planck Institute for Biological Cybernetics

Marc Ernst
Max Planck Institute for Biological Cybernetics

     Abstract ID Number: 105
     Full text: Not available
     Last modified: March 26, 2007
     Presentation date: 07/07/2007 11:30 AM in Quad General Lecture Theatre
     (View Schedule)

Abstract
A desirable goal in combining multimodal information is to maximize the reliability of the percept. This may be done using optimal strategies, where unimodal signals are weighted by their relative reliabilities. In real-world situations, stimulus conditions frequently change and with that the reliability of the signals. Here we ask whether the system takes such changes in reliability into account by adjusting the weights online in the McGurk effect.

Subjects were presented with an audiovisual recording of an actors’ face producing a series of syllables. Such syllables were composed either by consistent multimodal information, or by an auditory /ba/ and a lip movement of /ga/. Subjects' task was to continuously report which syllable they perceived. We varied the reliability of the visual information by changing the visibility of the face.

With inconsistent multimodal information, we found that increased reliability of the visual signal biased perception towards the illusory /da/ percept whereas decreased reliability biased perception towards the auditory /ba/. However, changes in reliability had perceptual consequences only after a delay of 3-4 seconds. From this we concluded that the reliability estimate of the sensory signals is not instantaneous but continuously updated with a time constant of a few seconds.