Evaluation of AI-Generated English Reading Comprehension Questions by Japanese Teachers of English

Abstract

In English education, reading comprehension exercises are one way educators can test a student’s understanding of a passage of text. Given the myriad skills which are used to comprehend text, creating good exercises is a very time-consuming process which prevents many educators from using reading comprehension more often. This paper utilized a generative AI model to create questions and answers on reading comprehension exercises and then conducted an empirical study targeting Japanese teachers of English. The surveys had the participants rank human-authored and AI-authored questions and answers on several different qualities and additionally collected the opinions of the participants regarding AI-assisted material creation. The results indicated that the questions created by the AI model still lag human-authored questions in several important respects but was still deemed as a useful tool to assist in question creation by the surveyed teachers.