Effect of System Utterances Conveying Understanding Results in Chat-Oriented Dialogue Systems

Abstract

In dialogue systems, conveying the understanding results of user utterances is important because it enables the users to feel understood by the system. However, it is not clear which types of understanding results should be conveyed to users; some utterances may be offensive, for example, and some may be too commonsensical. In this paper, we explore the effect of conveying the understanding results of user utterances in a chat-oriented dialogue system by an experiment with human subjects. We created system utterances conveying various understanding results and then investigated which types of results were favorable and unfavorable through manual evaluation. We found that understanding results referring to a user’s internal state were unfavorable, while those related to user’s positive attributes were likely to be favorable. In addition, understanding results related to objective facts about users or general facts unrelated to users were favorable. These findings can be used as a guideline for constructing a dialogue system that conveys appropriate understanding results.