Japanese Journal for Research on Testing
Online ISSN : 2433-7447
Print ISSN : 1880-9618
Volume 16 , Issue 1
Showing 1-6 articles out of 6 articles from the selected issue
  • Ayano Tsubota, Hidetoki Ishii
    2020 Volume 16 Issue 1 Pages 1-12
    Published: 2020
    Released: December 01, 2021
    JOURNAL FREE ACCESS

    Item writing guidelines have been examined in many studies. However, the way of functioning and effect sizes of each guideline are still uncleared. If flaws in items affect item characteristics such as proportion of correct under the situation that examinees do not aware of the flaw, it will ruin the measurement. To clarify these problems, 52 paired multiple-choice questions’ items on English, mathematics and language are developed and administered to 477 university students. In each item pair, one follows an item writing guideline and the other does not follow the guideline. As the result, it is found that proportions of correct are different in several item pairs although the flaws are not recognized by examinees. Proportions of correct of items which depend on other item or contain negativewording options become lower. And, these item flaws are rarely pointed out by examinees. Item writers should notice that item characteristics can be unintendedly affected by item writing.

    Download PDF (598K)
  • Sayaka Arai
    2020 Volume 16 Issue 1 Pages 13-30
    Published: 2020
    Released: December 01, 2021
    JOURNAL FREE ACCESS

    For multiple-choice items, it is required to measure not only knowledge but also the ability to think. Although it is evaluated that some items of the National Center Test measure the ability to think, do examinees actually answer them using the ability to think?

    In this study, I used some items of the National Center Test (World History), in order to investigate participants’ answering process of them. In [Survey 1], a questionnaire survey was conducted. In [Survey 2], descriptions of the actual answering process were collected and analyzed. Through [Survey 1] and [Survey 2], it is shown that the items which are evaluated as items to measure the ability to think are actually answered by inferring based on the contents of the items as being expected. It is also shown that even the items deemed as measuring knowledge were answered not only by knowledge but also by inference based on knowledge.

    Download PDF (734K)
  • Yoshito Tan, Kensuke Okada
    2020 Volume 16 Issue 1 Pages 31-44
    Published: 2020
    Released: December 01, 2021
    JOURNAL FREE ACCESS

    Cognitive diagnostic models (CDMs) are a class of statistical models that diagnose the mastery of respondents’ cognitive traits, which are called attributes or skills. In standard CDMs, mastery of attributes is represented by binary discrete parameters. However, attribute mastery can be better described by continuous parameter that represents the degree of mastery rather than binary mastery/nonmastery status. The probabilistic-input, noisy conjunctive (PINC) model, as well as its higher-order extension, were proposed as noncompensatory models that assumed continuous attribute mastery, relaxing the assumption of discreteness. In the present study, we propose the probabilistic-input, noisy disjunctive (PIND) model as well as its higher-order extension that apply the same idea, relaxation to continuous variables, to compensatory models. After showing parameter recovery performance by simulation study in several conditions, we report the estimation result of these models using real data. The paper concludes with a discussion of the applicability of the proposed models.

    Download PDF (616K)
  • Shin-ichi Mayekawa, Tatsuo Otsu
    2020 Volume 16 Issue 1 Pages 45-68
    Published: 2020
    Released: December 01, 2021
    JOURNAL FREE ACCESS

    So called “the loss of information” due to the use of graded score was investigated using both theoretic approach and numerical simulation. In theoretic approach, the standard error of estimation of the true score from graded score was derived within the framework of classical test theory. It was found that, when the reliability of the test is not extremely high, the loss of information is negligible. The similar result was obtained as the result of simulation study.

    Download PDF (421K)
  • Kumiko Shiina, Hirohito Sakurai, Kei Ito, Sayaka Arai, Hisao Miyano
    2020 Volume 16 Issue 1 Pages 69-86
    Published: 2020
    Released: December 01, 2021
    JOURNAL FREE ACCESS

    Differences in scores in the “Practical Reading” and “Mathematical Thinking” tests between admission categories are analyzed for freshmen in a university participated the monitoring survey. Although both of the tests are non-subject-based tests, the score distributions of the freshmen in admission category with a high weight of the academic achievement tests are higher than those of other admission categories. This suggests that the abilities cultivated based on the learning of subjects up to high school are reflected in the scores of the tests. The freshmen by admission on recommendation with basic test of Japanese language and English show a relatively high score distribution for the both tests. It is suggested that applicants’ preparation for the basic tests led to the development of the abilities to be measured by the “Practical Reading” and “Mathematical Thinking” tests.

    Download PDF (719K)
  • From the view point of The Principle of Prospective Student Protection
    Naoki T. Kuramoto, Tomohiro Miyamoto, Hiroyuki Nagahama
    2020 Volume 16 Issue 1 Pages 87-108
    Published: 2020
    Released: December 01, 2021
    JOURNAL FREE ACCESS

    In the university entrance examination of the 2021 academic year, the system will be changed on a larger scale than ever before. Correspondingly, Tohoku University released “notices” twice in 2018. Results of a high school survey conducted the previous academic year became a valuable basis for making decisions regarding changes to the university admissions. Therefore, to create a reference to use for making decisions for the 2022 academic year exam, we surveyed opinions on the basic policy of Tohoku University. This paper mainly reports a summary of the results of that survey, focusing especially on changes in opinion between the two surveys regarding the English tests provided by private testing companies, the introduction of which was postponed by the Ministry of Education. As a result, high schools expressed strong support for the basic policy of Tohoku University. The result drew strong attention to the power and responsibility of individual universities related to university entrance examinations.

    Download PDF (668K)
feedback
Top