Furoshiki Wrapping Corpus: A resource for embodied semiotics of instruction for operation

Abstract

Embodied semiotics aims to explicate the cognitive and social processes by which an actor's bodily movement becomes a meaningful "signal" when understood and responded to by interactant in everyday situation. The authors have been annotating and analyzing the "Miraikan SC Corpus," a multimodal corpus of interactions between science communicators (SCs) and visitors at the National Museum of Emerging Science and Innovation (Miraikan), with a focus on such characteristic body movements as moving and pointing. However, research on the aspects of how actors' bodies interact with physical objects in the material environment has been insufficient. Then, we designed and experimentally recorded the "Furoshiki Wrapping Corpus", in which learner's understanding of an instructor's physical action with objects can be observed through the learner's bodily motions. This paper outlines the design of the corpus, the procedure for sharing and releasing the data, and the characteristics of the recorded data.