The Impact of Agent Appearance and Task Success on User Psychology in Large Language Model-Based Agents

Abstract

This study investigates the psychological effects on users when interacting with text-based conversational agents that utilize large language models (LLMs), focusing on the agent’s appearance and the success or failure of tasks performed with the agent. The experiment involved a collaborative task between a human and an agent related to contest entries. A between-subjects experimental design was employed, featuring four levels of agent appearance and three levels of task success. A total of 297 participants recruited via crowdsourcing took part in the experiment. The results suggest that differences in the agent’s appearance affect users’ feelings of guilt for using the agent, and that task success influences perceptions of the appropriateness of the suggested text and the agent’s contribution to the outcome, as well as suggesting that both appearance and task success factors influence perceptions of the agent’s human-likeness.