Auditory-visual temporal discrimination: Evidence for usage of a temporal cross-correlator

Martin Banks, Vision Science Program, University of California, Berkeley

Abstract
The brain combines information from different senses in a way that approaches statistical optimality. The precision of the combined estimate is better than the precision from either sense alone (Ernst & Banks, 2002). For example, the nervous system combines appropriate auditory (speech train) and visual signals (lip movements) to achieve better word recognition (Grant et al., 1998). As part of examining how auditory-visual correlations in time guide signal combination, we measured temporal discrimination with auditory-auditory, visual-visual, and auditory-visual stimuli. Two brief stimulus pairs were presented on each trial. One had a longer inter-stimulus interval than the other and the observer indicated which was longer. We measured interval-discrimination thresholds as a function of the base interval. Auditory-auditory and visual-visual thresholds decreased with increases in base interval from 0 to 40 ms. With yet larger intervals, threshold rose proportional to the base interval (Weber’s law). The auditory-visual thresholds had the same dipper, but the lowest threshold occurred at much longer base intervals. We will show that the observed behavior is consistent with the use of a temporal cross-correlator with a longer sampling window in the cross-modal than in the within-modal case.

Not available

Back to Abstract