Typification of Knowledge Errors in Dialogue Response Generation Using Pretrained Large-Scale Language Models

Abstract

In recent years, with the development of neural dialogue systems, the framework of knowledge-enhanced dialogue has attracted much attention. In knowledge-enhanced dialogue, dialogue is conducted while obtaining knowledge from external resources such as knowledge graphs. However, a problem exists that the response sentences generated by the system may contain information that is contrary to the given knowledge. This problem has not been sufficiently studied and undermines the reliability of dialogue systems. In this study, we constructed a knowledge-enhanced dialogue model using a pre-trained language model, and investigated the nature and tendency of knowledge errors in the knowledge-enhanced dialogue task. As a result, indicate the existence of error patterns that have not been previously considered. Furthermore, we have gained insights that will contribute to improving the reliability of dialogue systems.