Audio-visual binding - hemodynamical and electrophysiological correlates in a congruency-task

Matthias Bischoff, Bender Institute of Neuroimaging and Department of Clinical and Physiological Psychology, Justus-Liebig-University, Giessen, Germany

Abstract
To investigate the percept of audio-visual binding, simple visual and auditory stimuli were presented synchronously or asynchronously. The ventriloquism effect – the localization of a sound is biased by and towards a simultaneous visual stimulus – was used as an indicator for the influence of crossmodal binding. fMRI and EEG were recorded during task performance in order to reveal activation in areas discussed to be involved in multisensory processing in the literature and to identify correlates of crossmodal binding in EEG-spectrograms.
Contrasting trials with reported ventriloquism-effect vs. the no-binding condition revealed activation in the right insula, superior temporal sulcus and parieto-occipital sulcus. Implementing the ventriloquism-effect allows us to relate these activations to the percept of consciousness-related binding processes. The parameter values extracted from EEG-spectrograms of the trials of the experimental condition were higher than those of the control condition. Using this extracted parameter as modulator of brain activation during task performance in the fMRI-analysis revealed an activation pattern similar to the one of the fMRI-analysis based on the ventriloquism-effect described above. Activation was observed in the insular, STS and POS and additionally in the superior colliculi and right intraparietal sulcus.

Not available

Back to Abstract