Effect of Violations of Uni-dimensionality on the Practicality of True Score Equating

Abstract

  This study investigates the practical consequence of true score equating (TSE) in the case of violation of the uni-dimensionality assumption, which is assumed to be the item-response model. We selected multidimensional ability, the ratio of the misfit item in the test form, the difference in scale between two forms to be equated, the number of items, and item discrimination as the independent variables. We selected the accuracy of the score conversion table derived from the TSE method and the accuracy of pass judgment as the evaluation indices. Results of the simulation study indicated that the score conversion table likely will exhibit errors that are non-negligible from a practical perspective if the test displays strong multidimensionality, a high ratio of misfit items, a large difference in scales between two forms, a large number of items, and high discrimination. Furthermore, regardless of the number of items, where item discrimination is high, the accuracy of judgment from the fit condition is greatest for the condition where ρ=.4, 50%. Under the same conditions, the results suggest that judgment accuracy is lowest when the equating coefficient is highly deviant.