The time course of multimodal percepts induced by reliability changes

Massimiliano Di Luca, Max Planck Institute for Biological Cybernetics

Abstract
A desirable goal in combining multimodal information is to maximize the reliability of the percept. This may be done using optimal strategies, where unimodal signals are weighted by their relative reliabilities. In real-world situations, stimulus conditions frequently change and with that the reliability of the signals. Here we ask whether the system takes such changes in reliability into account by adjusting the weights online in the McGurk effect.

Subjects were presented with an audiovisual recording of an actors’ face producing a series of syllables. Such syllables were composed either by consistent multimodal information, or by an auditory /ba/ and a lip movement of /ga/. Subjects' task was to continuously report which syllable they perceived. We varied the reliability of the visual information by changing the visibility of the face.

With inconsistent multimodal information, we found that increased reliability of the visual signal biased perception towards the illusory /da/ percept whereas decreased reliability biased perception towards the auditory /ba/. However, changes in reliability had perceptual consequences only after a delay of 3-4 seconds. From this we concluded that the reliability estimate of the sensory signals is not instantaneous but continuously updated with a time constant of a few seconds.

Not available

Back to Abstract