"Look OVER Hear;" Spatial Congruity and the Ventriloquist Illusion

Jeannette R. Mahoney, Cognitive Neurophysiology Lab, Nathan Kline Institute

Abstract
Visual stimuli can alter the perceived location of auditory stimuli. This so-called ventriloquist illusion has been attributed to the superior spatial acuity of the visual system over the auditory system. Here, we investigated the intensity of the ventriloquist illusion when spatial processing conditions are optimal for both auditory and visual stimulation (i.e., when spatial motion direction of both auditory and visual inputs can best be discriminated). Participants were presented with audiovisual (AV) compound stimulus pairs (AV1-AV2) in such a way that they experienced apparent motion for each of the auditory and visual stimuli. The simultaneously presented auditory and visual elements of AV1-AV2 pairs were either spatially congruent or spatially incongruent, and participants judged whether the auditory and visual stimuli moved in the same or different directions. As would be expected, participants perceived a high percentage of AV1-AV2 congruent pairs as congruous (90%). Surprisingly, they also perceived 75% of AV1-AV2 incongruent pairs as being congruous; thus, demonstrating a strong ventriloquist effect. The existence of illusory congruent trials indicates cross-modal binding mechanisms between auditory and visual inputs. Electrophysiological results from the same study are being processed to examine the neural correlates of this cross-modal illusion.

Not available

Back to Abstract