2012 Volume 14 Issue 4 Pages 467-476
We proposed an embodied video communication system in which a humantype avatar called "VirtualActor" which represents interactive behavior is superimposed on the other speech partner's video image in a virtual face-to-face scene. The effectiveness of a video communication system was demonstrated in an experiment of comparison with the scene in which a reduced own video image is superimposed on the other talker's video image using the picture-in-picture method. However, this system had some problems, such as the detailed adjustment of video images and the lack of portability of sensors. In this paper, we develop a headset-type motion-capture device which reflects the talker's head movements directly using an acceleration sensor and gyro sensor, and employ a CG character which moves based on talker's own motion and generates motion automatically based on the on-off pattern of talker's voice. Further, we propose the concept of an embodied video communication system in which the CG character is superimposed on the other talker's video image in a face-to-face scene, and develop a prototype called "E-VChat". A communication experiment is performed to confirm the effectiveness of the E-VChat system for 12 pairs of subjects using three communication modes: "Headset," "Headset + Generated motion automatically as a talker's avatar," and "Headset + Generated motion automatically as an talker's support agent." The results show that all communication modes tested are affirmatively assessed by sensory evaluation, and the "Headset + Generated motion automatically as a talker's avatar" mode is evaluated highly by a paired comparison. Finally, we develop a multiple-character E-VChat system using an audience that nods in response to the talker's voice, and confirm the effectiveness of the system in an interview-style communication experiment.
