McGurk effects in deaf subjects with cochlear implants

Julien Rouger, Cerveau et Cognition CNRS UMR 5549

Abstract
The "McGurk" effect demonstrates that visual information from lip movements profoundly modify the auditory perception of speech by normally hearing subjects. Using the "McGurk" effect to probe multisensory interactions, we have investigated whether similar patterns of interference extend to postlingually deaf patients using a cochlear implant (CI). Normal (n= 19) and cochlear implant subjects (Nucleus, n=19) were assigned to the presentation of visual, auditory and visuo-auditory congruent or non-congruent stop consonant speech syllables, and have to repeat what they had heard or seen. Phonetic categorization based on mode or place of articulation was performed on stimuli and answers. Results show that CI subjects present difficulties to associate auditory stimuli to their corresponding phonetic categories, with better performances for manner of articulation compared to mode (56% vs 42%). Visual stimuli produced similar phonetic categorizations between the two groups, with better performances for place of articulation (61% vs 36%). Visuo-auditory conditions induce higher performances for all categories (76% correct) as consequence of multisensory integration. Bimodal protocols (congruent and non-congruent) show that cochlear implant provides primarily cues for manner of articulation, whereas lip-reading provided cues for place of articulation. Our results suggest similar mechanism of complementarity between vision and audition in normal and CI subjects.

Not available

Back to Abstract