visual speech affects discrimination of syllables in the auditory cortex: an MMN study
Poster Presentation
Kislyuk Daniil
Cognitive Technology Group, Laboratory of Computational Engineering, Helsinki University of Technology
Möttönen Riikka
Cognitive Technology Group, Laboratory of Computational Engineering, Helsinki University of Technology Sams Mikko
Cognitive Technology Group, Laboratory of Computational Engineering, Helsinki University of Technology Abstract ID Number: 135 Full text:
Not available Last modified:
March 18, 2006
Presentation date: 06/21/2006 8:30 AM in Hamilton Building, McNeil Theatre
(View Schedule)
Abstract
Infrequent changes in the visual component of audiovisual speech stimuli, resulting in the “McGurk effect”, elicit a mismatch response (MMN) in the auditory cortex even when the auditory stream remains constant (Sams et al., 1991; Möttönen et al., 2002; Colin et al., 2002). This result has been interpreted to suggest that visual information modifies processing of auditory information in the auditory cortex. However, this MMN may arise due to the change in the visual stimulus per se.
In the present EEG experiment, we avoided this possibility by keeping the visual sequence constant. An odd-ball sequence of auditory stimuli consisting of frequent /va/ syllables (standards) and infrequent /ba/ syllables (deviants) was presented to nine subjects. Deviant stimuli in the unisensory acoustic stimulus sequence elicited a typical MMN, indicating stimulus discrimination. When the acoustic stimuli were dubbed onto a video of mouth constantly articulating /va/, the deviant acoustic /ba/ was perceived as /va/ due to the McGurk effect and was perceptually indistinguishable from the standards. Now acoustic deviants did not elicit any mismatch response, indicating that auditory cortex did not discriminate between the acoustic stimuli anymore.
Our finding suggests that processing visual information can change neural representations underlying auditory discrimination similarly as occurs when the properties of the acoustic stimulus are changed.
|
![](images/spacer.gif) |
Learn more
about this
publishing
project...
|
|
|