Binding of letters and speech sounds in the human auditory association cortex critically depends on temporal synchrony

Nienke van Atteveldt, Dept. of Cognitive Neuroscience, Faculty of Psychology, Maastricht University, The Netherlands

Abstract
Temporal proximity is a critical determinant for cross-modal integration by multisensory neurons, as is shown by single cell recordings in different animal species. Information content occurs as additional binding factor for multisensory information of increasing complexity. We used fMRI to investigate the relative importance of these different factors for successful multisensory integration of letters and speech sounds. For this purpose, we manipulated both temporal relation and information content (congruency of letters and speech sounds) within the same experimental design. Our results reveal that temporal relation and information content interact when causing fMRI responses to multisensory stimuli in the human auditory association cortex (planum temporale). In this region, multisensory integration was only observed when the information from the two modalities was delivered simultaneously. These results highlight the importance of timing in the efficient binding of learned relations between auditory and visual linguistic information. Furthermore, the strong resemblance of the revealed temporal windows in the planum temporale to those demonstrated by single cell recordings in animals support the hypothesis that basic integration rules apply to the binding of learned multisensory associations without natural correspondence. Finally, the present study shows the suitability of fMRI to study temporal aspects of multisensory neural processing.

Not available

Back to Abstract