手形状と物体形状の相関を利用した深層学習に基づく画像からの把持形状推定

抄録

In this research, we present a hand shape estimation method from a pair of color and depth images obtained from a RGB-D camera during object manipulation where a hand and an object are mutually occluded. In the proposed method, a depth image is segmented into hand, object, and background regions, and two-stream convolutional neural networks (CNN) are trained to classify the hand shape given a pair of segmented depth images of the hand region and the object region. Experimental results show that a high recognition ratio can be achieved by the proposed method compared with four different one-stream CNNs trained by color images, unsegmented hand and object region depth images, hand region depth images, and object region depth images.