Motion Generation for Home Robot by Using VAE-GAN + LSTM Based on Multi-Viewpoint Images During Operation of Household Items

Abstract

In this paper, we aim to apply motion generation method using machine learning based on human demonstration data to generation of task motion of home robot. A generation model by using VAE-GAN based on image data at task from multiple viewpoints is created, and a network model that can recognize the positional relationship between the robot hand and the object is also created. In addition, LSTM is learned by inputting latent variables generated by the image network model and joint angle data of the robot at each time, and network model capable of outputting the joint angle command value necessary for the task is created. Pushing task of the chair by the home robot is executed on the dynamics simulator by the learned network model and its usefulness is discussed.