2017 Volume 6 Issue 1 Pages 113-125
The raw scores obtained from the rating-scale method reflect not only the construct of interest in the test but also the response styles of the respondents. The method of anchoring vignettes was developed in order to distinguish between the two. A method for anchoring vignettes data based on the multidimensional item response theory (MIRT) was proposed recently; it has an advantage because it is based on the well-established modern test theory. The current study extends this framework in the following manner. First, an improved statistical model selection is introduced, based on the Watanabe-Akaike information criteria and leave-one-out cross-validation using pareto-smoothed importance sampling. Second, the Hamiltonian Monte Carlo estimation algorithm, which has a numerical advantage in complex models, such as the current one, is introduced. Third, two empirical datasets are comparatively analyzed using the proposed method. The results consistently indicate the utility of the bias-correction based on the anchoring vignettes and MIRT model. The study also discusses the importance of correcting the raw scores and usefulness of the MIRT-based model.
