Object Grasping Motion Generation by Attention Prediction Based on Language Instructions

Joint Research and Development of Hitachi, Ltd. and Waseda University

Abstract

In order to realize cooperative work between humans and robots, we developed a technology that understands human instructions and generates appropriate motions based on learning experience. In this paper, we proposed a motion generation method that uses an attention mechanism to extract object location information from visual information and an association mechanism to predict the indicated object. In a situation where multiple objects are placed, we confirmed that the robot accurately grasps the object indicated by a person.