A Vision Transformer-based handwritten alphanumeric characters recognizing intelligent pen

Abstract

Digitizing handwriting is mostly performed either by using offline methods such as Optical character recognition or using the combination of a special stylus and pad. This approach is costlier. Therefore, in this study, a deep-learning-based English alphabet and Arabic numeral character recognition method is proposed. A particular digital pen, equipped with an inertial sensor (three-axis accelerometer and three-axis gyroscope) and three force sensors, developed in our previous paper was utilized for this study. Vision transformer (ViT) which is drawing huge attention in the field of image classification and sequential tasks was adopted as a neural network model. The model achieved an excellent F1-score result of 0.993 during testing. This confirmed a promising capability of the developed digital pen for character recognition using neural network approaches.