Application of item response theory to a skill test by successful-trial approach

Abstract

The purposes of this study were 1) to examine whether Andrich's binominal model in which a single skill is measured by repeated trials is applicable to athletic skill tests, and 2) to develop a simple and practical evaluation method using it. This approach was applied to a basketball shot skill test. The test subjects were 203 male college students, who performed ten different basketball shots from various distances and directions. They attempted each shot five times, and the number of successful shots was recorded. In addition, the number of shots and goals for each student was recorded, and their basketball experience was surveyed. From the results, the following findings were obtained.
1) Regarding the one-trial per one-item approach, no one-dimensionality was found and the discriminative power was low, and therefore this approach was considered to be valid. In the approach using the number of successful trials, one-dimensionality, and a high correlation between the number of successful trials for each item and the total number of successful trials were found. This indicated that this approach has higher validity.
2) Concerning difficulty, it was found that a shot from under the goal post was easiest, followed by a running shot and a free throw, in decreasing order of easiness. A three-point shot was the most difficult.
3) The correlation between item parameters obtained from two-divided specimens was high, and sample cross-validation was confirmed although the correlation of ability estimates, θ, was low between the items divided into two. However, this does not necessarily conflict with the cross-validation of ability estimates because the number of items was too small to be divided.
4) Judging from the value of the information function, accurate measurement equivalent to 0.9 reliability was possible when the value of θ was between 0.0 and 2.5. That is to say, it is considered that the skill test is valid as a means of discriminating ability among subjects with average to above-average ability, like those used in the present study.
5) The value of ability estimates, θ, was significantly correlated with the number of shots and goals during the skill test, as well as the basketball experience of the students during their period at junior high school and high school. It is considered that the value is valid as a representation of the subjects' basketball shooting skills.
Finally, this study proposed an evaluation sheet, taking advantage of the fact that the number of successful trials can be a sufficient statistic of θ. This sheet will make it possible to hypothesize ability estimates easily and link the value to an anticipated number of successful trials. In this study, two cases were evaluated using the sheet.