Cross-Year Comparison of Test Score Distributions under a Testing Culture Characterized by Post-Hoc Disclosure of All Test Items

Linking Nationwide and Prefectural Tests

Abstract

This paper aimed to: 1) propose a means of cross-year comparison of test score distributions under a testing culture characterized by post-hoc disclosure of all test items, 2) consider its feasibility, and 3) examine its actual application collecting and analyzing real data. A practical method was proposed that enabled cross-year comparison by appropriately designing administration of nationwide and prefectural tests and linking their test scores. It was examined that the utility of the data, the need for collaboration between prefectures, and whether it was necessary to identify test taker identities. Finally, a comparison of the language test scores of ninth-graders over a three year time gap between 2006 and 2009 was conducted. Results indicated that there was little difference between test score distributions over time and what small differences there were, would be attributable to “application” not “knowledge”. Moreover, the feasibility of expanding our proposed data collection method over the number of tests, their versions, and regions of administration were also discussed.