Communication over the Internet using a 3D Agent with Real-time Facial Expression Analysis and Synthesis

抄録

We present a system for Internet communication that enhances traditional text-based chatting with real-time recognition and synthesis of the chat parties’ facial expressions. It is composed of three main modules: a real-time facial expression analysis component, a 3D agent with facial expression synthesis and text-to-speech capabilities—a talking head, and a communication module. So far we have realized a prototype to find out attractive ways of communication on the Internet and are currently experimenting how to utilize this new type of modalities in chat communication.