Exploring by eye and hand: bimodal interactions in the perception-action cycle
Single Paper Presentation

Nicola Bruno
Dipartimento di Psicologia, Università di Trieste

Marco Bertamini
School of Psychology, University of Liverpool

Georg Meyer
School of Psychology, University of Liverpool

     Abstract ID Number: 160
     Full text: Not available
     Last modified: March 19, 2006
     Presentation date: 06/18/2006 4:00 PM in Hamilton Building, Foyer
     (View Schedule)

Abstract
Perceiving the three-dimensional structure of an object often involves merging vision and haptics over extended periods of exploration. As exploration progresses, new information may require changing how the two sensory signals are treated. We asked whether (i) bimodal interactions during extended observation can adjust to changes in the quality of unimodal sensory signals; and (ii) there are specific temporal constraints on these adjustments. When viewing a three-dimensional Necker cube with one eye, participants experience illusory reversals even while they feel the cube with their hands. This property makes the visual-haptic Necker cube an excellent model to investigate bimodal processes when previously consistent bimodal signals change in quality over time or begin to conflict. Our participants reported reversals while we varied their exploratory behavior. We varied fixation by asking participants to move fixation from the front to the back of the cube, and we varied haptic information by asking them to hold the cube with only the fingers or to explore the cube with the hands. We recorded fixations as well as hand movements. Both kinds of exploration had a clear effect on the probability of experiencing alternative percepts as well as on the time course of reversals. In addition, when both were performed the effect of visual exploration was modulated by that of haptic exploration. Overall, these results suggest that (i) bimodal processes can adjust to changes in the quality of unimodal signals during exploration, and (ii) the adjustments caused by these changes are characterized by specific temporal dynamics.