Question Generation in Dialogue Systems based on Embedding Representation of Knowledge and Topics: Towards Food Preference Interview Systems

Abstract

With a goal of building a dialogue system that can acquire food preference of users through conversation, this study proposes a method for selecting topics and generating questions based on a large-scale knowledge graph, Freebase. We define a topic as a relation between two entities in Freebase and create a topic embedding model that learns the similarity between topics based on the Wikipedia corpus. This model is used to select topics related to the current one. Moreover, we create a knowledge graph embedding, which is used to predict and complement missing entities in Freebase. These proposed methods enable to generate questions about user preferences while expanding the topic widely. We developed a web-based text chat system that generates questions based on the proposed methods and conducted a user study using crowd workers. Results demonstrate that the system can continue a dialogue longer by expanding the topics from a single dish. We also investigated the quality of the questions generated by the system. In addition, we showed that subjects received a better impression of the variation of topics and the continuity of contexts when the dialogue failure occurrence was below a certain level.