Scoring Method for“Select All the Choices That are True”Items

Abstract

The multiple-choice (MC) format is the most widely used format in objective testing. The “select all the choices that are true” item is one variation of the MC format. This item has no instructions to indicate the number of correct choices. Although many studies have developed and compared scoring methods for this type of item, the results have often been inconsistent.

Most scoring methods that have been developed are based on the number of choices correctly selected. In this study, we treated the response patterns of examinees as binary variables and we proposed new scoring methods based on the similarity or the degree of association between response patterns and key patterns. Two proposed methods, the multiple true-false (MTF) method, and the negative marking (NM) method were compared and their characteristics were revealed. Among these methods, the Jaccard index method was considered to be appropriate from the viewpoint of score diversity and calculation simplicity. The results showed that the response patterns with high scores were basically identical for the methods.