Cross-modal Associative Networks: are we all synesthetes?

Jan B.F. van Erp, Department Human Interfaces, TNO Human Factors

Abstract
About 15 years ago, Marks and Melara did choice reaction time experiments with compound visual/auditory stimuli. In these experiments, observers had to respond to one stimulus dimension (e.g. brightness) while the second (i.e., pitch) was irrelevant. They found advantages for ‘trials accompanied by informationally irrelevant “matching” versus “mismatching” stimuli from the other modality’. In the present experiment, we extend the previous experiments by including the tactile modality. 12 students did a speeded discrimination task with compound stimuli constructed from two brightness levels (high / low), two auditory pitch levels (high – 135 Hz / low – 60 Hz) and two vibration frequency levels (high – 135 Hz / low – 60 Hz; matched for subjective intensity). We found significant effects (p’s < .01) in the conditions with the auditory stimuli as irrelevant modality only. Favourable effects on both the RT and the accuracy were present for matching stimulus dimensions, i.e., high pitch with high brightness or high vibration frequency, and low pitch with low brightness or low vibration frequency. This result underlines the assumption that the brain contains cross-modal associative networks in which not only place and time are associated across modalities but many other stimulus attributes, including pitch, brightness, colour, vibration frequency, and motion direction.

References
Marks, L.E. (1989). On cross-modal similarity: the perceptual structure of pitch, loudness, and brightness. J Exp Psychol Hum Percept Perform. Aug;15(3):586-602.
Marks, L.E., Hammeal, R.J. & Bornstein, M.H. (1987). Perceiving similarity and comprehending metaphor. Monogr Soc Res Child Dev. 52(1):1-102.
Melara, R.D. (1989). Dimensional interaction between color and pitch. J Exp Psychol Hum Percept Perform. 15(1):69-79.
Melara, R.D. (1989). Similarity relations among synesthetic stimuli and their attributes. J Exp Psychol Hum Percept Perform. May;15(2):212-31.
Melara, R.D., Marks, L.E. & Lesko, K.E. (1992). Optional processes in similarity judgments. Percept Psychophys. Feb;51(2):123-33.

Not available

Back to Abstract