Synaesthesia and the McGurk effect

Gary Bargary, Trinity College Dublin

Abstract
Synaesthesia is a rare heritable condition in which normal sensory stimulation gives rise to abnormal additional experiences. Much debate exists regarding the nature of the association between the inducing stimulus and the concurrent synaesthetic experience, particularly where in the hierarchy of processing the association manifests itself. We were interested in the question of whether in cases of multisensory integration, would either of the unisensory inputs induce a synaesthetic experience or would the integrated multisensory percept act as the inducing stimulus. To address this question we carried out an audiovisual integration task with 12 linguistic-colour synaesthetes, using the well known McGurk effect using real words. For example the sound of ‘bait’ incongruently matched with the viseme ‘gate’ leads to the McGurk illusion percept ‘date’. The experiment consisted of 3 conditions; audio only, visual only and incongruent audiovisual. The synaesthetes task in each condition was to first report what colour they experienced to each stimulus and then to report the word they perceived. The audio stimulus was correctly reported in 63% of audio only trials and correct viseme was reported in only 3% of the visual-only trials. In the audiovisual condition predicted McGurk illusions occurred in 72% of trials. Analysis of the colour data indicated that the colour reported invariably matched the colour of the integrated multisensory percept, i.e. the McGurk illusion trials and not either of the composite unisensory inputs for each synaesthete. Our results have implications for our understanding of the nature of the information that induces synaesthesia.

Not available

Back to Abstract