Learning Rules of Interaction by Segmenting Multimodal DataUsing MC-GP-HSMM

Abstract

Humans can learn rules of interactions by observing others' interactions. To learn various interactions flexibly, we believe such an ability is also important for robots. To this end, we proposed Coupled Gaussian Process-Hidden Semi-Markov Models (C-GP-HSMM) that enabled robots to learn rules of interactions by observing motions of two persons. However, not only motions but also multimodal information is used in the actual human interactions. Therefore, in this paper, we extend C-GP-HSMM into a method to learn rules from multimodal data. In experiments, we show the proposed model can estimate the rules of a game, which contains multimodal interactions.