ロボットの行動計画のための動画生成AIモデルの学習と制御システムへの応用

抄録

In the field of robotics, the systems for robot behavior planning using video have also been proposed to deal with a variety of environments and tasks generically. The merit of this system is that it allows for generic planning unrestricted by specific policies, by outputting the generation of robot actions from the vast knowledge inherent in foundation models as video, given text input. In previous studies, the verification of action generation using video was emphasized, with object manipulation simplified for testing on simulations. This is not enough the usefulness of robot behavior planning using video for pick and place tasks. In addition, the systems from previous studies are not open, and such systems have not been sufficiently discussed. In this paper, we first focuse on utilizing Robotis’s TURTLEBOT3 Waffle Pi and Mobile Manipulator(referred to ”Waffle”) to construct robot behavior planning system using video.