Feature-based post-attentive processing for temporal synchrony perception of audiovisual signals revealed by random pulse trains

Waka Fujisaki, NTT Communication Science Labs, NTT Corporation

Abstract
The visual motion or stereo perception with spatially dense random dot patterns is regarded as evidence of pre-attentive signal matching without explicit identification of matching features. In contrast, we found that audiovisual temporal synchrony perception, tested by means of synchrony-asynchrony discrimination, was nearly impossible with an analogous stimulus - a temporally dense random pulse train made of luminance and sound-amplitude modulations. This suggests that audiovisual temporal synchrony perception is a post-attentive feature-matching process. The following observations further support this hypothesis. (a) Removing high temporal frequency components from the dense pulse train, or reducing the pulse density, made the synchrony judgments easier. This suggests the importance of temporally sparse features. (b) Even for dense pulse trains, the synchrony judgment was considerably improved when the stimulus included temporally sparse distinctive stimuli - a sparse pulse train made of red flashes and high-pitch pips embedded in a dense pulse train made of white flashes and low-pitch pips. Another experiment using more complicated pulse sequences indicated that prior knowledge about a matching attribute improves the synchrony judgment. These results suggest that audiovisual temporal synchrony perception is based on temporally sparse features that are made salient by stimulus-driven processes or top-down attention.

Not available

Back to Abstract