BERT-assisted extraction of a supplementary knowledge for text generator from external dataset including unstructured information

Abstract

Advances in neural language processing technology make it possible to generate more natural speech in non-task oriented dialogues such as chatting. In order to achieve natural and diverse speech production, it is necessary to generate utterances not only by referring to the history of previous utterances, but also by referring to appropriate external knowledge. In addition to structured information used in task-oriented dialogues (e.g., price and access in travel guide dialogues), unstructured information represented by user-generated contents (e.g., review text from general users) is expected to be utilized as external knowledge. However, it is not always easy to extract appropriate external knowledge according to context under a mixture of structured/unstructured information. In this study, we investigated a knowledge selection method for speech generation using BERT. We took the travel guide domain as a case study and examined the input information for appropriate knowledge selection.