Tactile Data Estimation Based on Nail Color Image through CNN

Abstract

A method of recording the tactile sensation felt by an operator during works is effective to record and reproduce the skills of craftsmen and advanced techniques. In this research, we develop a sensor identifying the magnitude and direction of the force from the nail color change using the phenomenon, in which the nail color changes due to the applied force to the fingertip. Thus, we developed a prototype of a nail color sensor equipped with a miniature CMOS camera and a green LED, and collected a dataset of normal force and nail color image variation on index fingers using an electronic scale. The obtained dataset was learned by Convolutional Neural Network (CNN) to determine applied normal force from the images. As a result of simulations, the root-mean-square error (RMSE) was about 0.54 [N] in the range of 0-10 [N].