Development of multi-degree-of-freedom hand prosthesis cover with sensory recognition

Hand prostheses by upper limb amputees are primarily dependent on visual feedback owing to the loss of sensory function in their hand. Although previous researches have been conducted on the restoration of the sensory function of amputees and on the development of electronic skin and gloves for sensory feedback, the realization to apply the research results to commercial hand prostheses is still difficult. In this study, we designed and developed a hand prosthesis cover including a sensory recognitive function which closely mimics human hand skin and, resulting into a multi-degree-of-freedom (DOF) myoelectric hand prosthesis. The proposed cover was made of flexible silicon to mimic the human hand skin, which can measure a grip force of less than 50 N using a tactile sensing module. The tactile sensing module was developed using a force-sensitive resistor sensor, and solid silicone vacuum compression molding by embedding the sensor and wires inside the cover was introduced for the fabrication process. A developed finger module for multi-DOF myoelectric hand prostheses by imitating the anatomical structure and motion mechanism of a human finger was compared the performance of the developed cover with that of a commercial cover on the developed finger module of the myoelectric hand prosthesis. The metacarpophalangeal joint range of motion of the finger module with the proposed cover with a 1.5 mm thickness was measured from 0° to 60° and the flexion angular velocity was recorded as a value of 60°/710 ms, which are similar to those of the commercial cover. From the experiments, we found that the hand gestures and grip motions seem to be similar with the proposed and commercial covers. From the experiment, we can suggest that the developed cover with sensory recognition can be directly applied to multi-DOF myoelectric hand prostheses. Also, with a fast and simple commercialized process, widely usage for amputees with the developed hand prosthesis cover will be available.

the motor function of human hands but also have a similar appearance to the human hand. Nevertheless, the sensory feedback of the hand is not achieved with these commercial covers, hence upper limb amputees still experience difficulties in daily life.
Upper limb amputees depend on visual feedback for limb control because of the loss of sensory function in their hand (Biddiss et al., 2007). However, if the myoelectric hand prosthesis is provided with the sensory function of the hand, the user could grip objects quickly and accurately. Research on sensory function can be categorized into studies on sensory feedback and studies on sensory recognition. With regard to sensory feedback studies, research has been conducted on the recognition of the grip force of the hand prosthesis by tactile sensors and direct transmission of the force to the nerves of the amputated hand (Rossini et al., 2010;Raspopovic et al., 2014), and also on the stimulation of sensations in the remaining muscles of the amputated area or in other areas (Arieta et al., 2006;Saunders and Vijayakumar 2011). With regard to sensory recognition studies, Wettels et al. (2009) developed tactile sensing mechanisms using air pressure to measure the grip force of a prosthetic hand, and Berg et al. (2013) and Abd et al. (2020) developed tactile sensing mechanisms using force sensors and liquid metal tactile sensors, respectively. These studies demonstrated that the developed sensory recognition can be applied to myoelectric hand prostheses to effectively grip objects. However, because these hands have sensing mechanisms for sensory recognition installed on the palm, fingertips, or finger joints, they are more similar to robotic hands than human hands and are difficult to equip with a cover mimicking human skin.
Recently, research was conducted on electric skin mimicking the human hand skin. The electric skin is equipped with multifunctional sensors that can imitate the sensory functions of the skin surfaces. Kim et al. (2014) developed an electric skin for hand prosthesis that can sense deformation, pressure, temperature, and humidity. Hua et al. (2018) developed an electric skin with various sensory functions, such as temperature, in-plane strain, relative humidity, UV light, and magnetic field sensing. Zhao and Zhu (2017) developed a thermosensation-based electric skin with sensory functions for pressure, airflow, material, and temperature. These electric skins have sensing functions for various stimuli, such as temperature, pressure, and heat. However, most of the electric skins developed in previous studies were in the form of partial patches and could only be applied by wrapping over the entire body of the multi-DOF myoelectric hand prosthesis, causing limitations in the development into a cover that can actually be worn on the hand prosthesis to perform various hand gestures and grip motions.
In this study, we propose a cover with sensory recognition that overcomes the problems outlined above. The appearance of the proposed cover is similar to the skin of the human hand and it can be worn on a multi-DOF myoelectric hand prosthesis to perform various hand motions and grip motions. The proposed cover is made of silicone material to imitate the human hand skin, and a tactile sensing module based on a force-sensitive resistor (FSR) sensor is mounted on the fingertip to measure the grip force of the hand. In this study, the changes in signals according to the pressing force were measured to determine whether the grip force of the fingertip generated during general grip motion can be measured using a tactile sensor. In addition, to examine whether the developed cover with sensory recognition allows various hand motions and grip motions, the developed cover was worn on the finger module for a multi-DOF myoelectric hand prosthesis, and the finger flexion angular velocity and the joint range of motion (ROM) were measured.
The remainder of this paper is organized as follows. In Section 2, we discuss the development of the proposed finger cover with sensory recognition for multi-DOF myoelectric hand prostheses. In Section 3, we present the performance and wearability test results of the developed cover. Finally, in Section 4, we present the conclusions of this study.

Finger module for multi-DOF myoelectric hand prosthesis
In this study, a finger for a multi-DOF myoelectric hand prosthesis was designed by mimicking the anatomy and motion mechanism of the human finger. The finger is composed of the distal phalange (DP), middle phalange (MP), proximal phalange (PP), and metacarpal bone (MB). The finger joints consist of a proximal interphalangeal (PIP) joint and metacarpophalangeal (MCP) joint (see Fig. 1 (a)). The length of the designed finger from the end of the DP to the center of the MCP joint is 83 mm, which is close to that of a human finger.
The flexion and extension of the finger are reproduced by pulling the tendon fixed on the phalange during contraction and relaxation of the muscle. In this study, a motor (Maxon, Switzerland) was used as the muscle, and a wire was used as the tendon. The motor used for finger flexion was mounted inside the finger PP, and the wire was fixed to the worm wheel gear and MP. Finger flexion is performed by pulling the wire fixed to the worm wheel gear when the motor is driven. The PP of the finger is flexed at the MCP joint by the motor-bevel gear-worm gear, and at this time, the wire fixed to the worm wheel gear is pulled and the MP and DP are flexed at the same time. The finger extension is performed by a restoring spring mounted between the PP and MP, and finger flexion is performed by a motor-wire type tendon-driven mechanism with one DOF (see Fig. 1 (c)).

Finger cover with sensory recognition
In this study, a finger cover applicable to a multi-DOF hand prosthesis was fabricated. In general, the covers for cosmetic hand prostheses are typically fabricated as follows. First, the user's healthy hand is scanned using a 3D scanner. An amputated hand model is then fabricated using the scan data and a 3D printer. From this model, a positive model is made using silicone (Body Double, USA) and semi-rigid plastic (Smooth-cast 45D, USA). Then, a mold of the positive model is made using epoxy; liquid silicone (RTV3040, USA) is poured into the mold, and the model is cured to cover the cosmetic hand prosthesis. The cover proposed in this study was developed through a similar process, but a tactile sensing module was inserted at the end of the cover and wires were embedded into the cover through solid silicone vacuum compression. Whereas the cover for cosmetic hand prostheses is fabricated using liquid silicone, the cover proposed in this study was fabricated using solid silicone (Chlorosil 20 Shore A, Germany), which has a higher durability and elasticity. The solid silicone compression method for fabricating the cover is as follows. First, a multi-DOF myoelectric hand prosthesis model is manufactured using a 3D scanner and a 3D printer, and a positive model is formed using semi-rigid plastic. Then, solid silicone is attached to the fabricated positive model. In this step, the solid silicone can be molded into the desired thickness using a digital silicone roller. After the attachment of the first layer of solid silicone to the positive model, the tactile sensing module and wires are mounted on the fingertips. Then, the second layer of solid silicone is attached, and suction is performed using a vacuum pump. Suction is performed twice for 1 h and 30 min at a pressure of 20 mbar (see Fig. 2 (a)). After the suction is completed, silicone curing is performed. For curing, drying is performed in an oven drying machine at 60 °C for 6 h. After drying, the cover for the multi-DOF myoelectric hand prosthesis is completed. Fig. 2 (b) shows a finger cover with sensory recognition fabricated using this process.

Experimental method for grip force recognition and finger cover performance
The commercial multi-DOF myoelectric hand prosthesis is not equipped with a sensing module that can measure the grip force of the hand owing to limitations in weight, shape, and size. Instead, it employs open-loop control using the load of the motor generated when gripping an object. However, as the load generated on the motor is small when performing the tip grip motion and precision grip motion used when gripping small objects, the user cannot quickly and accurately grip objects by performing repetitive movements to grip small objects. Therefore, in this study, a cover that can recognize a grip force of 50 N or less is proposed to help users grip small objects accurately and quickly using tip grip motion and precision grip motion. The precision grip force of a normal multi-DOF myoelectric hand prosthesis is lower than 50 N (Belter et al., 2013). Therefore, a tactile sensing module capable of sensing a grip force lower than 50 N was installed inside the developed cover. The tactile sensing module is mounted on the tip of the finger in consideration of the part that contacts the object when performing fingertip gripping and precision gripping. The installed tactile sensing module uses an FSR sensor (Marveldex, Korea) with a diameter of Φ5.0 mm, thickness of 1.0 mm, and pressing force measurement range of 0-50 N.
The voltage change of the tactile sensing module according to the pressing force was checked to confirm the grip force recognition function. The grip force measurement of the tactile sensing module was tested using a universal testing machine (Instron 5565, Instron Inc.) (see Fig. 3) with a measurement range of 0-5 kN, and the measurement accuracy of the system was within ± 0.5 %. Figure 3(b) shows the tactile sensing module used in the experiment and the shape of the tip that transmits the pressing force to the module. The tactile sensing module has an FSR sensor installed between solid silicone. The thickness of the module was changed from 1 to 3 mm with a step of 0.5 mm. Several modules with different thicknesses were fabricated in order to select the optimized structure for the required specifications of the cover. The tip that transmits the pressing force to the module was fabricated in a circular shape with a diameter of 15 mm, considering the diameter of the sensor. The signal of the tactile sensing module was measured while increasing the pressing force from 0 to 100 N in increments of 2.5 N.
The flexion angular velocity and joint ROM of the multi-DOF myoelectric hand prosthesis are important factors when gripping objects or performing hand motions. The cover acts as a load when the myoelectric hand prosthesis grips an object or performs hand motions. Therefore, in this study, the flexion angular velocity and joint ROM of the prosthetic hand finger with the proposed cover were measured. Figure 4 shows the experimental setup for the measurement of the flexion angular velocity and joint ROM of the finger module with the cover. The flexion angular velocity was measured from the load generated by the motor at the start and end of the flexion and the time taken. For the joint ROM measurement, the flexion angle of the finger joints was measured by taking photographs of the flexion motion of the finger wearing the cover.

Flexion angular velocity and joint ROM of finger module
The finger module for the multi-DOF myoelectric hand prosthesis developed in this study has a large spatial limitation owing to the installation of motors, gears, and wires, as in other commercially available five-finger myoelectric hand prostheses. For this reason, a sensor to measure the angle of the MCP joint when performing the flexion could not be mounted. Therefore, to measure the ROM and flexion angular velocity of the finger in this study, photographs of the ROM of the MCP joint were continuously taken. In the case of the PIP joint, the flexion angle increased in proportion to the MCP joint flexion; thus, only the ROM of the MCP joint was measured. The ROM and flexion angular velocity of the multi-DOF myoelectric hand prosthesis finger without the cover were measured. The ROM of the MCP joint was measured to be 0°-60°, and the finger flexion angular velocity was measured to be 60°/580 ms. These results are similar to those obtained with a commercial five-finger myoelectric hand prosthesis (see Fig. 5).

Tactile sensing module experiment
In this study, we developed a tactile sensing module capable of sensing the grip force of a multi-DOF myoelectric hand prosthesis. The module is equipped with an FSR sensor that can measure grip force in the range of 0-50 N. Figure  6 shows plots of the voltage of five modules with thicknesses of 1.0, 1.5, 2.0, 2.5, and 3.0 mm as a function of the pressing force applied at a speed of 6 mm/s. It can be seen that the voltage of the FSR sensor changed by 1000 mV as the pressing force range changed from 0 to 100 N. The change in voltage of the tactile sensing modules with thicknesses of 1.0, 1.5, and 2.0 mm (1340, 1170, and 1050 mV, respectively) was greater than that of the FSR, while that of the tactile sensing modules with thicknesses of 2.5 and 3.0 mm (860 and 850 mV, respectively) changed less. This result indicates that the modules with thicknesses of 1.0, 1.5, and 2.0 mm are more suitable for cover fabrication. However, the tactile sensing module with a thickness of 1.0 mm was torn upon repeated application of the pressing force; thus, a module with this thickness is not suitable for cover fabrication.

ROM and flexion angular velocity of the finger according to cover thickness
While the developed cover has an esthetic advantage by mimicking the skin and shape of the human hand, it acts as a load for the motor function of the hand when worn on the myoelectric hand prosthesis. Therefore, in this study, finger and flexion angular velocity were measured. Figure 7 shows the test results of the MCP joint ROM with respect to the cover thickness. It can be seen that the MCP joint ROM of the finger with cover thicknesses of 1.0, 1.5, and 2.0 mm is 0°-60°, that of the finger with a cover thickness of 2.5 mm is 0°-51°, and that of the finger with a cover thickness of 3.0 mm is 0°-49°. Because the ROM of the finger module developed in this study is 60°, the cover thicknesses required are 1, 1.5, and 2 mm. Figure 8 shows the finger flexion angular velocity measured with different cover thicknesses. It can be seen that the flexion angular velocity of the finger with the cover thickness of 1.0, 1.5, 2.0, 2.5, and 3.0 mm are 650, 710, 840, 930, and 1050 ms, respectively. The flexion angular velocity of the developed finger module is 580 ms, which suggests that the flexion angular velocity decreases with increasing cover thickness.

ROM and flexion angular velocity of finger wearing commercial cover
We measured the grip force sensing characteristics, MCP joint ROM, and flexion angular velocity of the finger module with covers of different thicknesses. When the cover thickness was 1.0 mm, the voltage changed according to the pressing force up to 1340 mV; however, the cover eventually tore following repeated application of the pressing force. When the cover thickness was 1.5 and 2.0 mm, the voltage change was 1170 and 1050 mV, respectively, and no tearing occurred even when the pressing force was repeatedly applied. The ROM of the MCP joint of the finger wearing covers with thicknesses of 1.0, 1.5, and 2.0 mm was measured to be 0°-60°. These results are consistent with the developed finger ROM. In addition, with increasing cover thickness, the finger flexion angular velocity decreased, indicating that thinner covers are desirable for optimal flexion angular velocity. Covers with thicknesses of 2.5 and 3.0 mm had lower grip force sensing characteristics than the FSR sensor and resulted in an MCP joint ROM smaller than 60°. In addition, with these thicknesses, the flexion angular velocity of the finger was very low; thus, these thicknesses could not be used for cover fabrication. Therefore, it is concluded that covers for multi-DOF myoelectric hand prostheses should have thickness of 1.5 and 2.0 mm for optimal performance.
The MCP ROM and flexion angular velocity of the finger with the cover developed in this study and a commercial cover, I-skin (Ossur Inc.), with a thickness of 1.6 mm were compared. Figure 9(a) shows the results of the ROM of the MCP joint of the finger with different covers. It can be seen that the ROM of the MCP joint of the finger ranged from 0° to 60° in all cases. Figure 9(b) shows the results of the flexion angular velocity of the finger with different covers. It can be seen that the 1.5 mm-thick developed cover and the commercial cover had the same flexion angular velocity of 60°/710 ms. These results indicate that the thickness of 1.5 mm is optimal for the cover.

Cover for multi-DOF myoelectric hand prosthesis
In this study, to verify the performance of the developed cover, it was worn on the myoelectric hand prosthesis developed in our previous study (Jung et al., 2020). The developed myoelectric hand prosthesis (Jung et al., 2020) is driven by a motor with a 64:1 gear ratio, and the driving voltage is 3.3 V. The specifications of the developed hand, shown in Table 1, are similar to those of commercially available myoelectric hand prostheses. Figure 10 shows a 1.5 mm-thick cover fabricated in this study for a multi-DOF myoelectric hand prosthesis and capable of sensing grip force. The cover was fabricated using the solid silicone vacuum compression method, and tactile sensing modules capable of sensing the grip force were installed at each fingertip. Figure 11 shows photographs of the multi-DOF myoelectric hand prosthesis wearing the cover fabricated in this study performing various hand gestures and grip motions. The hand gestures performed are thumb up, OK, victory, and point, which are commonly used for daily communication, and the grip motions are power grip, hook grip, lateral grip, and precision grip. The photographs demonstrate that the myoelectric hand prosthesis wearing the cover fabricated in this study can perform various hand motions smoothly to the same level as the hand prosthesis wearing the existing commercial cover, indicating that the developed cover can be readily commercialized.

Conclusion
Recent research on sensory recognition and feedback systems has produced a sensory recognition module that measures grip force by attaching a tactile sensing mechanism to the fingertips, joints, and palm of the hand prosthesis, and a sensory feedback system that stimulates nerves or skin. The sensory recognition and feedback system proposed in this study (See Fig. 12) can be an important means to reproduce the functions of the amputated. We developed a silicone cover for multi-DOF myoelectric hand prosthesis fingers with a function of sensory recognition of the grip force, similar to human hand. The thickness of the cover was optimized as a value of 1.5mm and a tactile sensing module capable of measuring a grip force ranging from 0 N to 50 N was embedded inside the cover by a simple and fast fabrication process From the evaluation of the performance, the multi-DOF myoelectric hand prosthesis wearing the developed 1.5 mm-  (Belter et al., 2013) 75x160 --103.3/1 - Table 1 Multi-DOF myoelectric hand prosthesis specifications. thick cover shows a finger ROM of 0°-60° and a flexion angular velocity of 60°/710 ms, which indicate that the available hand functions with performances is very similar to those of existing commercial covers. Therefore, the cover with sensory recognition proposed in this study can be easily to be applied for a multi-DOF myoelectric hand prosthesis. The results of this study are expected to guide the development of high performance prosthetic hands for upper limb amputees. We can also suggest that a precise grip motion of a small object using sensory recognition function will be available with a sensory feedback system to transmit the grip force perceived by the sensory recognition function. Fig. 11. Results of performing various hand gestures and grip motions of the multi-DOF myoelectric hand prosthesis wearing the developed cover. Four hand gestures (thumb up, OK, victory, and point) and four grip motions (power grip, hook grip, lateral grip, and precision grip).
Fig. 12. Sensory feedback system composed of the sensory recognition system and sensory feedback system. The sensory recognition system measures the grip force of the hand prosthesis using the cover developed in this study. The sensory feedback system transmits the grip force to the skin via the vibrotactile and temperature sensors.