Pseudo Training Data Generation for Automatic Aggregation of Open-Ended Questionnaire Responses by Large Language Models

Abstract

Analyzing surveys utilizing open-ended responses to questionnaires is a valuable approach to elucidating respondents' perspectives and opinions, thereby gaining insights. However, the analysis of responses on a large scale necessitates a considerable amount of manual labor. Thus, this paper takes an approach of automating the analysis of open-ended responses using large language models. We have generated several types of pseudo data for training category classification models and evaluated the performance of the models trained on each dataset. Through this process, we examine the performance improvements of category classification models using the pseudo datasets automatically generated and annotated by large language models. Evaluation results show that, through several operations of pseudo open-ended responses, we improved the category classification performance against real open-ended responses from 77% to 83%.