ChatGPT’s Performance on the English Reading Sections of the Common Test for University Admissions in Japan

抄録

Since ChatGPT was launched in November 2022, discussion has been taking place over its use in education. It has both positive and negative effects depending on how it is used. This study investigated what score ChatGPT can obtain on the English reading sections of the Common Test for University Admissions in Japan. By examining answers from ChatGPT, it was expected that possible applications of ChatGPT in creating and analyzing English tests could be proposed. As materials, three tests from 2021 and 2022 were used (National Center for University Entrance Examinations, 2021a; 2021b; 2022). Tuned English prompts and GPT-4 were used for better output. ChatGPT scored 91.3 out of 100 points with three errors on average. Error analysis indicated that one question might need to be revised. Other errors suggested that ChatGPT might not be good at comparing numbers and making subjective judgements. This study implies the potential for using ChatGPT in checking and improving reading test questions.