Cross-modal Effects in Human Robot Interaction

Abstract

This paper reports cross-modal effects between different modalities regarding an entrainment in human robot interaction. The entrainment is the phenomenon that human's behavior is synchronized with robot's behavior. We argued whether the crossmodal effects between words and gestures could influence the entrainment; especially, in the relationship between deictic words and pointing gestures, it is investigated how the percentages of deictic words and pointing gestures that people use to refer to objects are changed by the combination of deictic words and pointing gestures that a robot uses to confirm indicated objects. According to our experimental results, participates used their pointing gestures more frequently when the robot used its deictic terms, and participates used their deistic terms when the robot used its pointing gestures. The results showed the cross-modal effects between words and gestures has potential to occur in human robot interaction. We believe these findings are available to robot's dialog design to elicit human's behavior that a robot wants.