Relationship Between Coherence of Sequential Events and Dialogue Continuity in Conversational Response

Abstract

Dialogue continuity is a dialogue system-evaluation metric that indicates how well the system engages its users. Response coherence considered an important factor in determining dialogue continuity. Herein, we propose novel methods of selecting coherent responses for a given dialogue context. These methods improve coherence and dialogue continuity using related-event pairs such as “be stressed out” and “relieve stress.” Two re-ranking methods are proposed. The first method estimates the coherence of event pairs in a dialogue by matching event-causality pairs that are statistically extracted from web texts. The second method estimates the coherence of response candidates for the dialogue context using a coherence model. The results of an automatic evaluation show that objective coherence at the word level was improved by these re-ranking methods. In contrast, in the case of human evaluation, subjective coherence at the response level was not improved by the first method; however, the dialogue continuity was improved. These results seem to be contradictory. We conducted a correlation analysis and a case analysis to clarify the relationship between subjective coherence and dialogue continuity. The results indicate that subjective coherence does not have a strong correlation with dialogue continuity. Further, dialogue continuity may be improved if an event related to the dialogue context is selected.