Customer Service Evaluation using Roundtable with Multiple Large Language Models

Abstract

This paper proposes a method for correcting store staff's customer service speech using multiple large language models (LLMs) to improve the quality of the speech of store staff in customer service. In addition, the staff's speech is scored and a commentary is generated to provide a basis for scoring. By correcting the staff's utterances and providing a quantitative evaluation, appropriate feedback can be provided to the staff. This study adopts a roundtable method called ReConcile as one of the methods to utilize multiple LLMs. The results of evaluation experiments showed that by refining the output of multiple LLMs with ReConcile, it was possible to modify the utterances more appropriately than a single LLM and to assign evaluation points that were closer to human senses.