3D hand pose estimation method using CNN and high-speed vision system

Abstract

We present a 3D hand pose estimation method works in a high frame rate with a high speed vision system and CNN. We employ a high-speed camera for a fast gray-scale image acquisition which enable us to recognize fast and small finger movements and a normal RGB camera for an accurate estimation. In the proposed method, the 3D hand pose is estimated by CNN with RGB images and the estimated pose is corrected at every frame using difference of pose estimated by CNN with gray-scale between adjacent frames. Firstly, we develop CNN models for RGB and gray-scale images, and the former was more accurate than the latter in a depth direction. Secondly, we verify our method through an experiment evaluating accuracy of differences of estimated poses between adjacent frames.