A proposal of annotation scheme for body movement based on participants' understandings

Abstract

In this paper, we introduce an annotation scheme for body movement based on participants' understandings in interaction. In previous studies of annotation scheme for building a dialogue system, labels which labelers put on a software were determined in advance. In our annotation scheme, we use a form of sentence to describe body movements and set two kinds of hierachical-related tiers; upper tier for interactional practice and lower tier for psysical practice. And we set meta-level tier in order to observe a relationship between modalities. We believe that it is useful information to detect similar expressions and sequential pattern of multimodal interactions.