Construction of Multimodal Learning Models Based on Integrating Stochastic Models

Abstract

In order to realize human-like intelligence artificially, large-scale models are required for robots to understand the environment using multimodal information obtained by various sensors equipped in robots. However, as the scale of models becomes large and complex, it is difficult to construct such models and to derive and implement the equations for their parameter estimation. To overcome this problem, we proposed a framework Serket that makes it easy to construct large-scale models and estimate their parameters by connecting small fundamental models hierarchically while keeping programmatic independence. In this paper, we construct the integrated models of the modules such as variational autoencoder, Gaussian mixture model, Markov model, and multimodal latent Dirichlet allocation, and then show that it is easy to construct the integrated models and their parameters are optimized by communicating between the modules by using Serket.