Japanese Journal for Research on Testing
Online ISSN : 2433-7447
Print ISSN : 1880-9618
Volume 3, Issue 1
Displaying 1-5 of 5 articles from this issue
  • Kozo Yanagawa
    2007Volume 3Issue 1 Pages 3-13
    Published: 2007
    Released on J-STAGE: June 08, 2022
    JOURNAL FREE ACCESS

    The purpose of this study is to examine the effects of item stem preview and/or answer option preview of multiple-choice listening comprehension tests on item difficulty and item discrimination. Three different item formats of MCQ listening comprehension test were addressed in the present study: (1)both preview (printed item stems and answer options provided prior to listening) (2)answer option preview (printed answer options provided prior to listening: printed item stems after listening) (3)item-stem preview format (printed item stems provided prior to listening; printed answer options after listening). Item format was referred as stems and/or option preview in this study.

    The result of the experiments revealed that item format of MCQ listening comprehension tests affects item difficulty. The mean score of the format (2)was found significantly lower than that of the format (1)or(3). This was primarily be attributed to the nine items on the format (2), which showed higher item difficulties than on the other two formats. No significant difference was found in item discrimination power across the three formats.

    Download PDF (5441K)
  • A problem of test information function
    Yasuharu Okamoto
    2007Volume 3Issue 1 Pages 35-47
    Published: 2007
    Released on J-STAGE: June 08, 2022
    JOURNAL FREE ACCESS

    A case of inadequacy of using information function in adaptive test was demonstrated by simulation. In adaptive test, assumption of independent and identical distributions is not satisfied, and in the beginning of the test, amount of data is not enough to insure asymptotic representation of distribution of parameter estimate by information function. The simulation showed that these conditions can induce contradictive discrepancy between the true distribution of parameter estimate and the distribution represented by information function. Adaptive test by minimum entropy criterion instead of information function was proposed. Minimum entropy criterion was adopted for psychophysical measurement by Kontsevich and Tyler (1999), and the method was named Ψ method. Behavior of adaptive test by minimum entropy criterion was checked by simulation, which demonstrated that the method by minimum entropy criterion with a uniform prior distribution works as well as the one by information function with respect to estimation of ability parameter, although it induces substantial conservative biases in case of a normal prior distribution. It was pointed out that as one of Bayesian approaches, the proposed method is worth further investigation to inspect its merits and demerits in various conditions and to exploit its multidimensional characteristics.

    Download PDF (1951K)
  • Ryuichi Kumagai, George Gotoh, Naoko Nakaume, Tadashi Shibayama
    2007Volume 3Issue 1 Pages 91-97
    Published: 2007
    Released on J-STAGE: June 08, 2022
    JOURNAL FREE ACCESS

    At Niigata University, a new curriculum of English education was introduced in the fiscal year 2005. As a result, the TOEIC IP test was given to all first-year students. In this research, the relationship between the results of the TOEIC IP test and results of the National Center Test for University Entrance Examinations (NCUEE) and "Standard English" was examined. Results indicated that there was a .62 correlation between the total TOEIC score and marks of the NCUEE. There was also a .32 correlation between the total score of the TOEIC and marks of Standard English. Moreover, the correlation between Standard English and each skill (or total score) of TOEIC varied depending on the teacher and the class, which had been streamed according to students' achievement. A regression analysis in which the NCUEE score was the independent variable and the total TOEIC score was the dependent variable, indicated coefficients of determination of.85 for teacher and .81 for class. It is suggested that these results can be used for selecting classes in terms of the teacher's characteristics or students' achievement.

    Download PDF (1026K)
  • Yasuo Miyazaki
    2007Volume 3Issue 1 Pages 123-146
    Published: 2007
    Released on J-STAGE: June 08, 2022
    JOURNAL FREE ACCESS

    Hierarchical Linear Modeling is one of the most frequently used statistical methodology for analyzing educational and developmental data which show a nested data structure, and it has been recognized that this methodology provides useful information when it is applied for analyzing large-scale educational data. However, it is unfortunate that researchers in Japan have not been exposed to this methodology much yet and thus not many studies have been conduced using this technique. There are two objectives for this article: One is to introduce this methodology to the researchers who wish to apply the technique to their research by providing two sets of typical analyses of organizational study and growth modeling using a National Education Longitudinal Study (NELS)data set. The other is to provide checkpoints of utilizing the results to educational policies from a standpoint of research design and statistical analysis.

    Download PDF (4409K)
  • Analysis of the “Admission Office Examination”
    Dai Nishigori, Naoki T. Kuramoto
    2007Volume 3Issue 1 Pages 147-160
    Published: 2007
    Released on J-STAGE: June 08, 2022
    JOURNAL FREE ACCESS

    University admissions have been considered as high-stakes selection processes in Japan. In consequence, fairness issues in university admissions have always been targets for social concerns. The concept of fairness, however, is not simple. It should be considered and treated as a complex psychological construct. In the present article, we analyzed subjective fairness perceived by university freshmen who had been university candidates until quite recently, from the viewpoints of fairness theories in social psychology. As a result, we obtained some noteworthy findings from the viewpoints of allocation principle on distributive justice, and rules on procedural justice. We also demonstrated the usefulness of social psychological fairness research. It is extremely important to point out the fact that there can be no ‘perfectly fair system’ everyone agrees. This fact implies that it is realistic to seek for reasonably fair situations that may minimize dissatisfaction in terms of fairness among people in concern.

    Download PDF (2052K)
feedback
Top