Japanese Journal for Research on Testing
Online ISSN : 2433-7447
Print ISSN : 1880-9618
Volume 17, Issue 1
Displaying 1-6 of 6 articles from this issue
  • Kenichi Kikuchi, Naoko Nakaune
    2021 Volume 17 Issue 1 Pages 1-7
    Published: 2021
    Released on J-STAGE: December 01, 2021
    JOURNAL FREE ACCESS

    In recent years, many universities have adopted the adjusted median method as a way to adjust scores between the optional subjects on university entrance examinations. However, there has been little discussion of its nature from a test theory perspective. Therefore, we examined the kinds of adjustment results that are produced by using the adjusted median method to adjust scores according to the difference in mean and standard deviation between the optional subjects. Results indicated that when test scores are distributed in normal distributions, the adjusted median method generally adjusts the score so that subjects which originally have a high mean distribution with a large standard deviation are more advantageous to the top performers.

    In principle, score adjustment should be carried out to eliminate such inequity. Meanwhile, the adjusted median method is advantageous in that even after adjusting the score, the zero and perfect scores remain unchanged, and the adjusted score can be easily calculated by using the median. The advantages and disadvantages of this method must thus be considered when being implemented.

    Download PDF (546K)
  • Hiroyuki Noguchi, Atsuko Osumi, Ryuichi Kumagai, Megumi Shimada
    2021 Volume 17 Issue 1 Pages 9-23
    Published: 2021
    Released on J-STAGE: December 01, 2021
    JOURNAL FREE ACCESS

    In this study, we examined the possibility of applying the CEFR to Japanese language education from the perspective of the scale construction theory. Specifically, our study was carried out as follows: ⅰ) Based on the illustrative descriptors specified by the CEFR for each communicative activity ('listening', 'reading', 'speaking', 'writing', and 'interaction'), we developed a questionnaire in which learners self-assessed the degree to which they 'can do' specific communicative activities in Japanese and in which they provided answers according to a 4-point rating scale. ⅱ) A survey of Japanese learners was conducted in Japan and overseas countries. ⅲ) Based on the data of this survey, the descriptors were put on the IRT scale by GRM (Graded Response Model), and the order of difficulty of the descriptors for each communicative activity was compared with that of the original CEFR. As a result, it became clear that in order to utilize the CEFR in Japanese language education, modifications and supplements are necessary, especially in the area of 'interaction', so as to reflect the differences of levels, the need for inclusion of unique elements of the Japanese language, and the sociocultural differences between Europe and Japan.

    Download PDF (754K)
  • -Using Language Test of the University of Tokyo Entrance Examination-
    Hiroyuki Masukawa, Hajime Shirouzu, Moegi Saito, Shinya Iikubo, Takuya ...
    2021 Volume 17 Issue 1 Pages 25-44
    Published: 2021
    Released on J-STAGE: December 01, 2021
    JOURNAL FREE ACCESS

    Aggressive reading is a prerequisite cognitive activity in college life, in which readers deconstruct a text into pieces of information and restructure them into a coherent understanding. University entrance examinees, however, often rely on shallow processing including test wiseness. We therefore developed CBT reading items to elicit aggressive reading for previous language tests of the University of Tokyo entrance examination. We compared this revised version with the original version transplanted from the examination to study the effects on items requiring knowledge integration using two groups of students, totaling 79, who differed in their experiences of aggressive reading. A two-way ANOVA revealed significant interactions between the versions and the groups. CBT also enabled process analyses that revealed a hidden difference in the solution processes of the two sub-groups of students who demonstrated equivalent good results. The results indicate the potential of CBT in assisting and assessing the reading processes.

    Download PDF (1357K)
  • Kentaro Fukushima, Nao Uchida, Kensuke Okada
    2021 Volume 17 Issue 1 Pages 45-59
    Published: 2021
    Released on J-STAGE: December 01, 2021
    JOURNAL FREE ACCESS

    Cognitive diagnostic models (CDMs) have been attracting attention as a method for diagnosing the learning subgoals mastered by examinees. In particular, CDMs for multiple-choice item assessments have been developed to elicit more information on examinees when they choose an incorrect option. However, the CDMs for multiple-choice items require test developers to specify a Q-matrix for each option of the items. Owing to the difficulty of developing such multiple-choice tests as well as the unavailability of a public dataset, existing studies have often conducted only simulation studies. Nevertheless, they have not examined their application to real data. Hence, knowledge regarding the empirical performance of CDMs for multiple-choice items is limited. To address this problem, this study composed a multiple-choice assessment of English with a Q-matrix and evaluated the performance of a CDM for multiple choices. The results imply that, although the model passed the predictive check, the examinees’ answering process to multiple-choice items could not be fully captured by the model, indicating the need for further model development.

    Download PDF (983K)
  • -A Comparative Case Study of Standardized Tests in Japan, China, and South Korea -
    Hongyu Nan
    2021 Volume 17 Issue 1 Pages 61-74
    Published: 2021
    Released on J-STAGE: December 01, 2021
    JOURNAL FREE ACCESS

    In 2020, the university entrance examinations of Japan, China and South Korea are entering a revolutionary transforming period. The new university entrance examinations reformation saw various prevention measures against COVID-19 being implemented. In this study, COVID-19 prevention measures for university entrance examinations taken by Japan, China and South Korea were studied and compared. Through comparison, it was found that Japan, China, and South Korea were either conducting or planning to conduct their national unified entrance examinations in spite of the corona virus pandemic. In China, examinations were postponed until July, one month behind usual, whereas for South Korea, examinations were postponed until December, behind by two weeks. Japan’s examinations were scheduled to be held in January, but the examination period has been extended. In this paper, the similarities and differences of measures taken by each country will be discussed under the perspectives of “ensuring fairness” and “determining relevant abilities”.

    Download PDF (824K)
  • An Analysis of English Test Preparation Materials for University Entrance Examinations in Japan
    Shotaro Baba
    2021 Volume 17 Issue 1 Pages 75-93
    Published: 2021
    Released on J-STAGE: December 01, 2021
    JOURNAL FREE ACCESS

    In the context of high-stakes testing, test takers often use a test preparation material called a “past exam paper.” Although this term is used to refer to a series of questions distributed on the past examination day, no study has given its operational definition. The present study examined the contents of English test preparation materials for university entrance examinations in Japan, and proposed four definitions: (a) an original entrance examination paper which has been distributed on the examination day; (b) a test preparation material which is composed of original entrance examination questions, model answers, and trend analysis of questions, all of which are published per university per year; (c) per university per subject; and (d) per subject. With these definitions, this study explored the reason why previous studies could not reach consensus on the effectiveness of a past exam paper in terms of fairness of test preparation and validity of test score interpretation.

    Download PDF (1011K)
feedback
Top