Cross-modal Recognition of Human Body Pose: Interaction of Motor and Visual Information

Yasuyuki Inoue, Department of Knowledge-based Information Engineering, Toyohashi University of Technology

Abstract
Recognition of human body is a special case of object recognition, since human body is very familiar and can be encoded with motor (self-body movement) as well as visual information. We aimed to investigate the effects of visual and motor encoding of body poses on visual recognition. Half of the subjects learned 16 different poses (8 front views and 8 back views) with visual observation. The others learned the same poses with visual observation and body action in that they imitated the presented posture by own body. 4 repetitions of the learning session were followed by a visual recognition test, in which a learned pose and a novel pose were presented and the subject judged which was learned one. We varied the view difference (0-180deg) of the learned poses between the learning session and the recognition test. For visual-encoding subjects the recognition performance with front-view encoding was significantly better than with back-view encoding, but for visual-and-motor-encoding subjects there was no difference. It is suggested that the visual encoding of human body has advantage in front-view encoding while the motor encoding has advantage in back-view encoding. Thus, we can learn novel postures view-independently by using both visual and motor information.

Not available

Back to Abstract