The Impact of an Interview Agent's Relationship-Building Dialogue Strategies on Language Performance

Abstract

The Oral Proficiency Interview (OPI) is a common test format used to assess the speaking abilities of second language learners. However, human interviewers exhibit individual differences and varying skill levels, making it difficult to maintain fairness and reliability across test-takers. To address this issue, there is growing interest in using AI agents to achieve standardized interviews. Nevertheless, the impact of AI agents' behavior on test-takers' psychological states and actual speaking performance has not been thoroughly investigated, which may affect the validity of the test. In particular, there has been little examination of how rapport-building conversational strategies used by AI agents - such as small talk, empathetic responses, and positive feedback - affect test-takers' intrinsic factors and performance. This study aims to investigate the impact of these rapport-building conversational strategies implemented by AI agents on test-takers' performance, engagement, and rapport in the conversation.