Identifying Handwork with Machine Learning Data Sets from Sensors Built into Gloves

Abstract

To digitalize handwork that remains at manufacturing sites, we developed a glove with built-in sensors that capture fingertip pressure and work sounds. To create an identification model with small amounts of training data, we developed a handwork identification model based on machine learning, in which a hidden Malkov model (HMM) extracts features from time-series information of sensor data and a support vector machine (SVM) identifies the type of handwork. The developed gloves and model identify specific handwork every 0.01 seconds. We experimentally confirmed that our proposed model can distinguish two typical types of handwork—connector insertion and screw tightening with a pistol grip electric screwdriver—with 82% accuracy from only 20 seconds of training data for each. This technology can be applied to recording handwork evidence and automation of incorrect work detection.