Purpose: We conducted a meta-evaluation of item characteristics such as percentage of correct answers, item discrimination degree and item difficulty using classical test theory (CTT) and item response theory (IRT) for 90 items in a gerontological nursing that created a computer based nationwide common achievement test (CBT) at nursing universities.
Subjects and Methods: We conducted a written monitoring survey by convenience sampling 730 college juniors prior to starting their on-site training at 23 public and private nursing universities. We analysed the percentage of correct answers by CTT, biserial correlations of Item Total (IT), item difficulty by IRT(2PL model), item discrimination degree and factor loading. From these results, we constructed item characteristic curves to evaluate each item's characteristics.
Results: The average percentage of correct answers for gerontological nursing was 65.8–69.3%. The correlation coefficients were
r = 0.30–0.41 for pharmacology and
r=0.28–0.38 for anatomy and pathology (
p<0.01). There were large gaps for both item difficulty (-5.851–4.068) and item discrimination degree (0.292–2.218). As a result, the information content curves exhibited each item's characteristics. The item discrimination degrees for 3 out of 90 items were low, which presents a subject for future study.
Conclusion: This study suggests that the remaining 87 items were considered to be usable in CBT however, 71.1% of the items had negative item difficulty, which indicated that they were easy items to answer. There remain some challenges in discriminating highly capable examinees in this CBT items.
View full abstract