Evaluation of an Interviewer Response-Generation Model for Eliciting User-Food Preferences Considering Semantic Content

Abstract

Obtaining users' preferences during a dialogue is desirable to provide personalized services. We collected interview dialogues aimed at acquiring food preferences, and created a response generation model based on the intention and semantic content of the interviewer's utterance by fine-tuning GPT-3.In this study, we investigated the performance of the proposed model by comparing with ground truth interviewer utterance, Zero-shot ChatGPT and a fine-tuned GPT-3 model that directly generates only response sentences as baselines. The subjective evaluation showed that in terms of eliciting the interviewees' food preference, the proposed model's response sentences were superior to those of the baseline models and comparable to real human interviews.Analysis of the characteristics of the response revealed that the proposed method 1) frequently generates questions in various dialogues and 2) produces more detailed and context-related questions compared to ChatGPT.