Attempts to improve the analysis of questionnaire surveys using large-scale language models

Abstract

Large-scale language models (LLMs) have spread rapidly since the release of ChatGPT, and many research reports have been published on the ability to understand and predict the mental state of others, such as their intentions, beliefs, desires, and knowledge, as represented by ToM (Theory of Mind). While the ToM field is expected to be applied to civil engineering fields such as urban planning that takes into account people’s personality, behavior, and feelings, there is a problem in that it is difficult to collect data on behavior and feelings and personality data at the same time. In this study, we prepared a dataset by simultaneously conducting a questionnaire and personality diagnosis, and verified whether LLMs can estimate the personality of the questionnaire respondents. Although the accumulated data is still not large, at around 200 people, it was confirmed that the personality traits of the respondents can be estimated with a certain degree of accuracy even from relatively short sentences such as questionnaire surveys, and it became clear that there is a high possibility of using LLMs in fields such as the analysis of human behavior and psychology.