The effects of Q-matrix misspecification in a cognitive diagnostic model on the diagnostic accuracy

Examination by considering cognitive hierarchy structures

Abstract

Cognitive diagnostic models(CDMs) are useful to assess student’s attribute mastery patterns but misspecification of a Q-matrix could lead to inaccurate measurement. The purpose of this study was to assess how misspecifications of a Q-matrix with cognitive structures called attribute hierarchy structures (AHSes) affects student’s attribute mastery patterns estimates and item parameter estimates by simulation experiment. We got four main results: 1. misspecification declined students’ mastery pattern accuracy estimate for all AHS, especially divergent structure, 2. overestimate condition had more effect than underestimate condition, 3. we could relax the effect if Q matrix had fewer misspecification, 3. sample size did not have effects on misspecification’s results, 4. misspecification would affect right setting item parameters in some AHS.