Solvers of Japanese Text Comprehension Questions Using Surface Clues

Abstract

This paper reports progress from 2014 to 2015 on development of solvers of Japanese comprehension questions in university entrance exam. Target questions are the multiple-choice questions in the essay section (Question No.1) in Japanese Language (Kokugo) of National Center Test. In 2014, we introduced a new scoring function using clause boundaries, which are automatically detected by our newly developed tool. The score of a choice is calculated as the average clause-similarity between the choice and a selected part of text body. In 2015, we developed a machinelearning based method, which uses seventeen features to determine the answer. They includes surface-similarity based features, clause-similarity based features, and choice-discriminative features. In addtion to the first formal run of Torobo Project in 2013, we participated in the two formal runs in 2014 and 2015; We were only a participant who submitted the result in Contemporary Japanese Language until now. After the 2015 formal run, we conducted an experiment using 276 questions to compare all developed solvers with various parameters. The best performance was obtained by a 2015 solver, which produced 117 (42%) correct answers. For the subset of 56 previous official questions in National Center Test, a 2014 solver was the best, which produced 32 (57%) correct answers. However, there is no statistical significance between the best 2015 solver and our first solver developed in 2013.