ヒューマンインタフェース学会論文誌
Online ISSN : 2186-8271
Print ISSN : 1344-7262
ISSN-L : 1344-7262
特集論文「若手研究者6」
Motion Inbetweening Based on Body Parts Integration for Sign Language Generation
Nabeela KhanSihan TanKatsutoshi ItoyamaKazuhiro Nakadai
著者情報
ジャーナル フリー HTML

2024 年 26 巻 4 号 p. 431-442

詳細
抄録

Sign Language Generation (SLG) is one of the emerging fields in Sign Language (SL) communication technologies. SLG employs a Sequence to Sequence (Seq2Seq) model with SL glosses serving as intermediaries. While using Seq2Seq is an optimal way of generating SL, it overlooks the fineness and smoothness of the inbetween transitions for SL glosses. In this work, we address the issues of fine motion prediction and smooth transition generation to seamlessly integrate glosses for natural SL expression in SLG termed as Sign Language Inbetweening (SLI). In SLI, we propose the Body Parts Integration (BPI) for fine motion prediction and the Variational Autoencoder-Generative Adversarial Network (VAE-GAN) generator with a progressive embedding for smooth transition generation over variational transition lengths. Evaluation of the proposed BPI-VAE-GAN model is conducted on a Motion Capture dataset of Japanese Sign Language performed by a native signer. Assessment employs Mean Angular Error (MAE) for fine motion prediction and mean Jerk for smooth transition generation. Results indicate that the proposed architecture significantly reduces MAE, outperforming the GAN by 46% reduction and considerably outperforms the linear interpolation. Additionally, Jerk analysis illustrates the system’s capacity to generate smoother motion sequences compared to GAN.

著者関連情報
© Non-Profit Organization, Human Interface Society
前の記事 次の記事
feedback
Top