Automatic Perceiving Topic Induction Evaluation and Utterance Selection for Target-Guided Dialogues

Abstract

In target-guided conversation, it is crucial to improve the user experience by leading the conversation toward the system's own goal without making the user feel guided and without making them aware of the system's objective.In this study, we propose SBIS-TGC (Surprisal-Based Induction Score for Target-Guided Conversation), an automatic evaluation metric designed to assess the degree of induction in system utterances, with the objective of achieving conversation goals without the user noticing either the system's goal or its guiding behavior.SBIS-TGC quantifies the sense of induction by calculating surprisal between utterances using an external language model.Through dialogue experiments employing a system that selects utterances based on SBIS-TGC, we demonstrate that the proposed method can reduce the perceived induction in target-guided dialogue and enable conversations where users remain unaware of the system's intended target.