2023 Volume 50 Issue 2 Pages 131-146
Cognitive diagnostic assessments (CDAs) require a large number of itemsto measure the target attributes with high precision. An automatic itemgeneration (AIG) system would help to reduce the cost and effort of itemwriting in CDAs. This study aimed to develop a valid AIG system for CDAsin linear equations of mathematics by designing an AIG system andexamining two aspects of a generated CDA in cognitive diagnosticmodeling: (a) the M-matrix, which specifies the set of attributesrequired by each item model and (b) the item discrimination index, whichis computed from item parameters in the deterministic input,noisy-and-gate (DINA) model. First, we compared an original M-matrix totwo alternative M-matrices by using information criteria, posteriorpredictive model checks, and item discrimination indices. Second, weexamined the magnitude and variability of the item discriminationindices for every item model. No substantially large differences werefound among the results from all of the M-matrices. The discriminationindices tended to be high in items that measured major attributes, andthe variabilities of the indices were small within each item model,except for a few item models. These findings indicate the validity ofour M-matrix and AIG system. Furthermore, they suggest ways to improvethe AIG system. Research limitations and how future studies can help toenhance the AIG system are discussed.
