Facial Expression Synthesis System Using Speech Synthesis and Vowel Recognition

Abstract

Herein, we report on the development of a system for agent facial expression generation that uses vowel recognition when generating synthesized speech. The speech is recognized using the Julius high-performance, two-pass large vocabulary continuous speech recognition decoder software system, after which the agent’s facial expression is synthesized using preset parameters that depend on each vowel. The agent was created using MikuMikuDanceAgent (MMDAgent), which is a freeware animation program that allows users to create and animate movies with agents.