Dialogue Analysis Based on User Proficiency in Collaborative Working Environment

Abstract

Recent advancements in Large Language Models (LLMs) have facilitated AI agents' ability to collaborate with humans, even in complex environments. However, the development of agents that adapt to user proficiency remains limited. Previous research introduced a dialogue-based agent designed to cooperate with humans in a virtual environment inspired by the cooking game Overcooked. This agent, however, relied solely on unilateral instructions from the human player, hindering effective collaboration with users possessing varying levels of proficiency, knowledge, and experience. In this study, we collect and analyze dialogue data from human-human collaborative work in order to develop an agent capable of adapting to users based on their proficiency levels. Participants with different levels of knowledge and skills are paired in groups of two to perform tasks in the game environment. Based on the collected data, we investigate how players of different proficiency levels adopt dialogue strategies and build cooperative relationships.