A Speech Dialogue System Utilizing Voice Activity Prediction and Objective Evaluation of Naturalness

Abstract

With the advancement of natural language processing technologies, dialogue systems that handle continuous speech are becoming increasingly prevalent. In particular, the responses of dialogue systems that provide backchanneling can disrupt natural conversation due to delays in response speed and interruptions during speech. However, evaluating these systems is challenging because it is difficult to separate backchanneling from the main dialogue. In this study, we focus on turn-taking to achieve natural interactions that include backchanneling, and we have developed a dialogue system utilizing Voice Activity Projection (VAP). This system predicts the start and end times of conversations, allowing for the distinction between backchanneling and interruptive speech. Experiments have confirmed improvements in naturalness, indicating its effectiveness for future dialogue system development.