Visual Servoing Corresponding to Various Obstacle Placements and Target Object Shapes Based on Learning in Virtual Environments

Abstract

In this paper, we propose an end-effector positioning method that can handle various obstacle placements and target object shapes. In the proposed method, two Convolutional Neural Networks (CNNs) are used to obtain ideal movement and avoidance movement, then these outputs and other conditions such as movable ranges of each joint are used to calculate a final movement by means of Quadratic Programming (QP) method. First, training data is collected in a virtualized environment in the physics simulator. This reduces the load on the actual experiment. On the other hand, the same environment is also used while the robot is actually performing visual servoing. This method enables visual servoing that is not affected by changes in the texture of a real environment. We confirmed that the proposed method was successful even if the target object is largely hidden by obstacles.