Japanese Journal for Research on Testing
Online ISSN : 2433-7447
Print ISSN : 1880-9618
Volume 5, Issue 1
Displaying 1-9 of 9 articles from this issue
  • Makoto Sano
    2009 Volume 5 Issue 1 Pages 3-21
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    Local item independence is generally a strong assumption for applying item response theory. Chen and Thissen (1997) proposed two models of local item dependence. One of the two is surface local dependence (SLD) which typically affects overestimations of discrimination parameters. This study evaluates the performance of some local item dependence indices focusing on the SLD condition. Simulation study was performed and computed local dependence indices with jIRTNew (Tsai & Hsu, 2005b) 0.35. The study suggests that the local item dependence indices applying mutual information are promising for detecting SLD and overestimations of discrimination parameters.

    Download PDF (1020K)
  • Yuichi Kawata, Manabu Iwasaki
    2009 Volume 5 Issue 1 Pages 41-51
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    In this study, we consider a situation where students who obtained low scores in their test results are given a remedial class and administered an after-remedial test to check the learning effects of the class. A beta-binomial distribution is assumed for the model of the test scores, namely, the number of correct answers in a test consisting of n questions. In addition, we consider the more common situation where the after-remedial test consists of m questions. We present the expectation value and variance of the score of the after-remedial test and the difference from the before-remedial test. Moreover, a degree of incompleteness in the before-remedial data is classified into three situations: selection, censoring, and truncation. For each situation, practical estimation procedures of the beta binomial distribution parameters are provided to fit the model by using the moment estimators. In every situation, we find that the statistical test adjusted for the regression-to-the-mean effect is appropriate. Thus, the result suggests that the application of proper tests is important for assessing the learning effect.

    Download PDF (842K)
  • Yiping Zhang
    2009 Volume 5 Issue 1 Pages 53-64
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    This research is about the influences on the measurement effect from item form and its testing method. As a result of comparison between the Short-Answer form with the Multiple-Choice form, the item difficulty parameters of Short-Answer form were found to be higher than Multiple-Choice form, but no clear differences were shown in the item discrimination parameters between these forms. Moreover, it was found that the measuring ability of the test hardly changed when replacing the item form. Furthermore, it was found that when the suitable alternatives were provided, the Multiple-Choice form was expected to show higher discrimination power than the Short-Answer form. It was also found that when using testing methods with more grades of scoring, the item information will be increased.

    Download PDF (815K)
  • –Discussing methodological problems and applying a polytomous model–
    Satoshi Usami
    2009 Volume 5 Issue 1 Pages 65-79
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    Neural test theory (NTT) is a data analysis method that has gradually become popular in educational measurement and psychometrics. In NTT, examinees are clustered to a discrete latent rank. The author noted several technical issues for future researches of NTT, such as accuracy of estimates for latent ranks and ICRP, consistency of estimates for latent ranks over item sampling, comparison among several optimizing criteria, and construction and improvement of algorithm for NTT models. In the present research, the author performed a simulation study by using polytomous NTT for ordered data, to compare consistency of estimates for latent ranks over between NTT and another method using total test score. Finally, a real data example for essay test data was shown by using polytomous NTT, and the author compared these results with methods based on item response theory and total test score.

    Download PDF (898K)
  • the perspective on designing appropriate interview
    Dai Nishigori
    2009 Volume 5 Issue 1 Pages 81-93
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    The present study investigated the psychological mechanism of those who applied to the interview examination which was executed on university admission from the view “structural factor” and “social factor” on procedural justice indicated by Nishigori(2007) . The result of analysis using SEM(Structural Equation Modeling) showed that the influence of “social factor” upon impression of interview composed of “accepting the rule of interview” and “sense of achievement” which they experienced were greater than “structural factor” involved the procedure of interview. In addition, the cognition to “fairness or justice on interview” or “the affirmative on interview” which people who have interviewing experiences have is far higher compared with that of people who have no interviewing experiences.

    Download PDF (694K)
  • Relationships between Proficiency Tests and the Ibaraki University Internal English Test
    Chisato Saida, Kunihiko Kobayashi, Hiroyuki Noguchi
    2009 Volume 5 Issue 1 Pages 95-105
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    Proficiency tests such as TOEFL or TOEIC have been utilized in curricular innovations in many universities. This research attempted to categorize the functions of the use of proficiency tests in university English education. Then, the practical use of an internal test in the new English curriculum in Ibaraki University was focused on. Ibaraki University has conducted the internal English test accompanied by a textbook since 2005. This research examined the criterion-related validity of the internal test. The correlation coefficient between the scores of the internal test and the National Center English Test was about .65, that of TOEIC IP was .65, and that of TOEFL ITP was .62. Correspondent score sheets were developed. As a result, the usefulness of the internal test increased.

    Download PDF (710K)
  • Ryuichi Kumagai
    2009 Volume 5 Issue 1 Pages 107-118
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    In this paper, computer programs we have developed for IRT analyses for beginners will be discussed. In developing, we attached importance to following two points: a. they have GUI which are easy to use intuitively, b. they are Freeware and everyone can easily obtain it.

    To verify the validity of the numerical results, we compared our programs with the existing programs. Then, it was shown, that numerical results in our programs were appropriate.

    Download PDF (724K)
  • Taketoshi SUGISAWA, Teruhisa UCHIDA, Kumiko SHIINA
    2009 Volume 5 Issue 1 Pages 127-135
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    We investigated the relationship between scores from the National Admission Test for Law Schools (NATLaS) and several tests measuring various abilities or traits in order to clarify what the NATLaS measures. Our results show that NATLaS scores well correlate with those from a basic logical thinking and a vocabulary test while there are low correlations between NATLaS scores and those from a questionnaire about attitudes toward critical thinking. The “Reasoning and analytical abilities” part of NATLaS correlates more strongly with the skills tested in the logical thinking test, and the “Reading comprehension and expressiveness” part correlates more strongly with the test of vocabulary. These results suggest that each part of NATLaS accurately measures examinee’s abilities as intended. Furthermore, a follow-up using results from another year’s NATLaS shows that these results are reasonably consistent.

    Download PDF (821K)
  • Yasuko Nogami
    2009 Volume 5 Issue 1 Pages 145-164
    Published: 2009
    Released on J-STAGE: May 31, 2022
    JOURNAL FREE ACCESS

    The effect of item exposure rate on changes in item properties and the quality of proficiency estimation is one of the most important concerns for practitioners of computerized adaptive testing. This article investigates the effects of high item exposure rate and repeated item presentations to the same examinee on item properties and proficiency estimation. In this study, I focused on the Computerized Assessment System for English Communication (CASEC), which is a commercially available computerized adaptive test. Simulated as well as the real data were analyzed and compared in terms of percentage of items answered correctly, item exposure frequency, and proficiency estimates of examinees. The results suggest that the effects of item exposure on item pollution and proficiency estimation were not so serious in the case of CASEC.

    Download PDF (869K)
feedback
Top