Stetho Touch: Touch Action Recognition System by Deep Learning with Stethoscope Acoustic Sensing

抄録

Developing a new IoT device input method that can reduce the burden on users has become an important issue. This paper proposed a system Stetho Touch that identifies touch actions using acoustic information obtained when a user's finger makes contact with a solid object. To investigate the method, we implemented a prototype of an acoustic sensing device consisting of a low-pressure melamine veneer table, a stethoscope, and an audio interface. The CNN-LSTM classification model of combining CNN and LSTM classified the five touch actions with accuracy 88.26%, f-score 87.26% in LOSO and accuracy 99.39, f-score 99.39 in 18-fold cross-validation. The contributions of this paper are the following; (1) proposed a touch action recognition method using acoustic information that is more natural and accurate than existing methods, (2) evaluated a touch action recognition method using Deep Learning that can be processed in real-time using acoustic time series raw data as input, and (3) proved the compensations for the user dependence of touch actions by providing a learning phase or performing sequential learning during use.