Construction and Evaluation of News Commentary Chat Corpus

Abstract

Expressing personal impressions contributes to the liveliness of dialogue in open-domain conversations. However, generating natural impressions of topics or other utterances requires an understanding of the conversation subject and the utterance of the interlocutor, as well as the utilization of common-sense knowledge, making it a challenging task for open-domain chatbot systems. We aim to develop an open-domain chatbot system capable of generating appropriate impressions in conversational contexts by incorporating impressions of real people as external information. In this study, we constructed a “News Commentary Chat Corpus,” enabling open-domain chatbot systems to learn the selection of suitable impressions and response generation based on these selected impressions. The proposed corpus comprises 1005 triplets containing “news articles,” “impressions of the people toward news articles,” and “dialogues on news articles.” Each dialogue was collected using the Wizard of Oz method, in which the system speaker engages in conversations by incorporating impressions written in posts on social media. The results of training systems to generate responses on this corpus using impressions of the people as external information revealed that the systems produced natural responses to context. Additionally, these systems generated a considerable number of responses that included impressions, thus enhancing the overall liveliness of open-domain conversations.