Reinforcement Learning for Selecting Types of Inverse Kinematics Solution of Collaborative Robots and Avoiding Singularity and Obstacle

Abstract

This paper proposes a method of avoiding singularities and obstacles by selecting the types of inverse kinematics solution of a manipulator by reinforcement learning. Deep Q-Network (DQN) selects from eight different types of solution that enable the manipulator to avoid singularities and obstacles throughout its motion path. This proposed method is applied to a 6-DOF collaborative robot. DQN, a type of reinforcement learning, is constructed with six joint angles as observation and eight types of solution as action. The motion path of the manipulator is divided into steps every 0.1s, and the type of solution at each step is selected by DQN. The agent is rewarded when the manipulator reaches the end of its motion path, and punished when it collides with the obstacle or itself, and according to the six joint angular velocities. As a result, DQN selects the types of solution that can avoid singularities and obstacles. The proposed method makes it possible to select which of the types of solution can realize the motion path of the robot hand without colliding with obstacles and which minimize the joint angular velocities.