Systematic review of automated scoring for divergent thinking tasks

Abstract

Recently, researchers have proposed a more valid process for performing and scoring divergent thinking (DT) tasks and new approaches to automatically score DT tasks using natural language processing and machine learning. Despite the recent proposals for this approach, its usefulness has been gaining attention. However, few comprehensive reviews have been conducted on the measurement advances in DT tasks in Japan. Therefore, this study summarizes issues related to scoring validity from the implementation of DT tasks and then conducts a systematic review focusing on the automated scoring approach for DT tasks. The current study examined (1) the types of participants, (2) the types of DT tasks, and (3) the types of automated scoring methods used for DT tasks. The results indicated that the automated scoring of DT tasks was conducted primarily on English responses to the language versions of alternative-use tasks, primarily for college students and adults in general. The most common automated scoring method is semantic distance based-scoring. However, scoring methods using supervised learning and large-scale language models have been proposed recently. Future research should examine the feasibility of using automated scoring methods by scoring DT tasks with more diverse participant populations and responses from speakers of more varied languages.