Estimation of the the opening of a translucent bag by integration of visual and tactile senses

Abstract

We propose a method to estimate the position of the opening center of a translucent bag by combining visual and tactile information, which is difficult to estimate accurately only by vision. We verify the proposed method using a 5-DOF arm-type robot equipped with a camera and a tactile sensor on its hand. In the visual estimation, the probability of the opening center of the bag is output as a probability distribution on the image from the input RGB image. In the tactile estimation, the position of the opening center is estimated from the tactile information obtained, and the area where the opening center of the bag cannot exist is estimated. By integrating these estimation results, the position of the opening center is estimated.