A Listener-Aware Speech Guidance That Adaptively Changes Speech Timing

Abstract

We aim to realize an automated spoken guidance system that monitors listener’s response tokens such as backchannels and fillers, and adapts its the behavior to them. Such a system is expected to improve the efficiency of the explanation, and reduce the user’s mental workload. As long as backchannels are detected regularly, the system continues to explain. Constrastively, if backchannels are not detected for a certain period of time, the system confirms the user’s understanding. In addition, when a filler is detected, the system stops talking immediately and waits for user’s utterance. In order to realize the system, we worked on real-time detection of listener’s response tokens and its integration into a dialogue system. To confirm the effectiveness of the system, an interaction experiment was conducted. The experiment was designed to compare the proposed listener-aware system that adapts its behavior according to listener’s response tokens, with a system that does not adapt to the listener. The result suggested that the adaptive behavior of the listener-aware speech guidance influenced users’ strategies of social signaling to artifacts. It also showed a greater variability in the listener-aware system’s pause length and shorter explanation time, depending on the user’s level of understanding. On the other hand, no positive effect of the proposed system on the user’s level of understanding was observed.