Unsupervised Segmentation for Video Using Convolutional VAE and Gaussian Process

Abstract

Humans recognize perceived continuous high-dimensional information by dividing it into significant segments such as words and unit motions. We believe that such unsupervised segmentation is also an important ability for robots to learn topics such as language and motions. To this end, we have been proposed the Hierarchical Dirichlet Processes-Variational Autoencoder-Gaussian Process-Hidden Semi-Markov Model (HVGH) which is composed of a deep generative model and a statistical model. HVGH can extract features from high-dimensional time-series data by VAE while simultaneously dividing it into segments by Gaussian process. In this paper, we propose a method that can segment not only high-dimensional time-series data but also videos in an unsupervised manner by improving VAE of HVGH to Convolutional VAE. In an experiment, we used a first-person view video of an agent in the maze to demonstrate that our proposed model estimates more accurate segments than the baseline method.