Inference of Potential User Needs and Interactive Navigation based on GPT

Abstract

An interactive navigation system for VirtualHome has been proposed to guide users to places and things they need by inferring their potential requirements from their utterances based on commonsense and home environment knowledge graphs. In the previous work, the cost of constructing knowledge graphs and dialogue rules was high. We propose a system that has the same functionality as previous work using GPT (GPT-3.5 and GPT-4). We designed prompts for navigation and inference of users' potential requirements by extracting the relationships between rooms and objects in the VirtualHome environment. We also designed prompts to determine the timing of guidance based on utterances. By providing these prompts to GPT, we realized the main modules in the proposed system. Using the evaluation datasets, we compared and evaluated the proposed system with the system of previous work. We found that the proposed system outperformed the accuracy of the system of previous work.