Item Response Theory (IRT) was originally developed for creating tests in multiple-choice form, and recently has become more popular in psychology and sociology. In sport science, IRT is used especially for pass-or-fail tests of motor-skill performance. This communication presents a more fundamental understanding of IRT in relation to sport science and associated areas, and proposes an application of IRT to pass-or-fail tests of motor-skill performance. Our conclusions are: (1) Reliability has to be investigated from an objective viewpoint, or from the stability of pass-or-fail measures, but not from the standard error of estimated ability. (2) Greater use of the ordinal scale model of IRT is recommended instead of a dichotomous one for a pass-or-fail test, because a person's rating cannot be strictly dichotomous; it must be graded. (3) The two-parameter model, or the Rasch model, is recommended because in a motor-skill performance test, fewer parameters are stable, and the level of certainty is different from that of any written test. (4) Items with differential item-functioning should not be included in a test, because they cannot measure ability fairly. (5) Although missing values are sometimes found in motor-ability measures and observations, no attempt to fill in the values based on likelihood should be made using incomplete data sets. More attention must be paid to the method of collecting data.
