Development and Evaluation of a Question Creation Support System That Automatically Generates Questions and Answers Based on Past Exams

Abstract

In this study, we aimed to reduce the burden of creating questions in certification exams by developing a question creation support system that automatically generates questions and answers based on knowledge extracted from questions and answers in past exams. In addition, we developed a prototype of the system for the Morioka history and culture expert certification called “Moriken”, which is sponsored by the Morioka Chamber of Commerce and Industry. This system is characterized by questions and answers that are automatically generated using a knowledge base constructed by extracting knowledge contained in past exams. Results of a preliminary experiment with students and an experiment on use of the system by the “Moriken” question creation committee demonstrated that the system could automatically generate questions that could be used with minor modifications in the certification exam. The mean time spent on each question was significantly shorter when using the system than when not using the system. Moreover, 78％ of the participants in the preliminary experiment and 60％ of the participants in the usage experiment reported that the burden question creation was reduced.