Binding of letters and speech sounds in the human auditory association cortex critically depends on temporal synchrony
Poster Presentation

Nienke van Atteveldt
Dept. of Cognitive Neuroscience, Faculty of Psychology, Maastricht University, The Netherlands

Elia Formisano
Dept. of Cognitive Neuroscience, Faculty of Psychology, Maastricht University, The Netherlands

Leo Blomert
Dept. of Cognitive Neuroscience, Faculty of Psychology, Maastricht University, The Netherlands

Rainer Goebel
Dept. of Cognitive Neuroscience, Faculty of Psychology, Maastricht University, The Netherlands; F.C. Donders Centre for Cognitive Neuroimaging, Nijmegen, The Netherlands

     Abstract ID Number: 72
     Full text: Not available
     Last modified: March 18, 2005
     Presentation date: 06/07/2005 9:00 AM in MART Auditorium
     (View Schedule)

Abstract
Temporal proximity is a critical determinant for cross-modal integration by multisensory neurons, as is shown by single cell recordings in different animal species. Information content occurs as additional binding factor for multisensory information of increasing complexity. We used fMRI to investigate the relative importance of these different factors for successful multisensory integration of letters and speech sounds. For this purpose, we manipulated both temporal relation and information content (congruency of letters and speech sounds) within the same experimental design. Our results reveal that temporal relation and information content interact when causing fMRI responses to multisensory stimuli in the human auditory association cortex (planum temporale). In this region, multisensory integration was only observed when the information from the two modalities was delivered simultaneously. These results highlight the importance of timing in the efficient binding of learned relations between auditory and visual linguistic information. Furthermore, the strong resemblance of the revealed temporal windows in the planum temporale to those demonstrated by single cell recordings in animals support the hypothesis that basic integration rules apply to the binding of learned multisensory associations without natural correspondence. Finally, the present study shows the suitability of fMRI to study temporal aspects of multisensory neural processing.