Learning Integration Concept of Objects and Motions Based on Segmentation and Classification of Time Series Multimodal Information

Abstract

Humans acquire various kinds of information with minimal cognitive load by using concepts and this capability makes it possible to deal with unknown events. Therefore, we consider that concepts also make it possible for robots to predict unobserved information from observable one. We define concepts as categories formed by clustering the perceptual information. In this paper, we propose a method for estimating a temporal segments of multimodal information based on co-occurrence relationship among modalities and classifying them into categories in an unsupervised manner. In the experiment, in which object images and robot's motions obtained by robot's manipulating objects were used, concepts that represents the co-occurrence relationship between objects and behaviors were formed.