THE ASSUMPTION OF THE EQUAL PROBABILITY OF CHANCE SUCCESS

Abstract

The “multiple-choice” form of test, in which answer alternatives are given, has frequently met with the objection that an examinee who does not know the correct answer to the item can make a substantial score by chance success. Several methods have been proposed to correct the effect of chance success on items and test statistics. These methods are all based on certain assumptions concerning chance success, which were as follows:
(1) An examinee who is ignorant of the correct answer to the item will fail in the free answer form of test, and he will choose the answer at random from the alternatives in the multiple-choice form.
(2) All alternatives in an item have the equal probability of chance success, that is, the probability is the reciprocal of the number of alternatives.
This study aims at investigating the validity of the above assumptions.
Methods: Two parallel mathematical tests consisting of 12 items were prepared, one of which was the “control test” (free answer form), and the other was the “experimental test” of five different forms the 2-alternative form (Form A), the 3-alternative form (Form B), the 4-alternative form (Form C), the 5-alternative form (Form D), and the free answer form (Form E), These tests were given to 3rd grade pupils (N=2, 100) of some secondary schools during October and November, 1964. Ss were divided into two groups:
Group 1: A control test was given in the usual way, but the experimental test was given without showing the problems and the examinees were asked to make a pure guesswork on the answer sheets.
Group 2: A control and experimental tests were given in the usual way.
Results: In the pure guesswork of Group 1, the percentage of response was different among alternatives and the mean score was significantly different from the chance score, contrary to the assumption of the equal probability of choice (Tables 1 and 6-7).
The results of Group 2, on the other hand, did not contradict the assumption of chance success. Score obtained were compared with the estimated scores which consisted of the true score and the chance score, and good agreement was obtained. All this is probably due to the fact that the proportion of the chance score in the observed score was so small as to make the difference between the observed score and the estimated score negligible.