IIEEJ Transactions on Image Electronics and Visual Computing
Online ISSN : 2188-1901
Print ISSN : 2188-1898
ISSN-L : 2188-191X
Weakly-Supervised Learning for Continuous Sign Language Word Recognition Using DTW-Based Forced Alignment and Isolated Word HMM Adjustment
ジャーナル 認証あり

2019 年 7 巻 2 号 p. 88-96


The reduction of the manual work of annotation is an essential part of sign language recognition research. This paper describes one weakly-supervised learning approach for continuous sign language word recognition. The proposed method consists of forced alignment based on dynamic time warping and isolated word hidden markov model adjustment using ‘embedded training’. While the proposed forced alignment only requires one manual annotation for each isolated sign language word, it can generate sufficient quality of the annotation to initialize isolated word hidden markov models. ‘Embedded training’adjusts initial hidden markov models to recognize continuous sign language words using only ordered word labels. The performance of the proposed method is evaluated statistically using a dataset that includes 5,432 isolated sign language word videos and 4,621 continuous sign language word videos. The averaged alignment error of the proposed forced alignment was 4.02 frames. The averaged recognition performances of the initial models were 74.82% and 91.14% in the signer-opened and trial-opened conditions, respectively.Moreover, the averaged recognition performances of the adjusted models were over 65.00% for all conditions.The evaluation shows significant improvements compared to the previous weakly-supervised learning.

© 2019 The Institute of Image Electronics Engineers of Japan
前の記事 次の記事