Abstract
A translation test is one of the most common reading test methods in Japan, although its reliability and validity have been quite controversial. This study investigated the reliability and validity of translation tests as a measure of reading comprehension in Japanese university entrance examinations, with a particular focus on two research questions: (a) In terms of reliability, what types of translation materials cause difference in rating severity? (b) In terms of validity, what types of sentence can make great differences between translation and comprehension tasks? In order to examine the first research question, four experienced teachers scored English-to-Japanese translations made by 102 university students. Results showed that rating divergence is attributable to raters' different points of scoring (holistic vs. partial), and raters' leniency for inappropriate Japanese expressions or careless mistakes. The second research question was examined by administering 18 sets of translation and reading comprehension tasks to the same participants as above. Results showed that 4 out of 18 English sentences used in Japanese university entrance examinations were considered to be inappropriate as materials for translation tasks when examinees' reading comprehension was intended to be measured.
