LLMを利用した不明を含むYES-NO疑問文の返答に関する意図推定手法の比較検討

抄録

In dialogue systems, it is especially important to ask questions that can be answered with either “yes,” “no,” or“I don’t know” intentions (YES-NO questions) in order to confirm the user’s intentions and status, and to accuratelyinterpret the intentions of the user’s answers. In this study, we aimed to perform highly accurate intention estimationfor generic topics, and to be able to determine unknowns as well. Specifically, we created a question answering corpus(Japanese yes-no question-and-answer pairs), designed multiple intention estimators using a large-scale languagemodel, and compared and evaluated their accuracy. As a result, the GPT model with an additional all-combininglayer (Fine-tuned GPT) showed the highest estimation accuracy, achieving 91% accuracy. On the other hand, whenprompt programming was performed using GPT-4 (Few-shot learned GPT-4), we observed a possibility that there wasa tendency to judge as unknown responses for which intention estimation was difficult. The results of this study areexpected to provide valuable guidelines for future research and practical use, as they suggest a policy for selecting anestimation method and tuning a model in the intention estimation task.