2011 Volume 7 Pages 1-7
Purpose: To test the reliability of methods used to evaluate the refusal skills of middle school students pressured to smoke or drink by comparing the ratings given by physical education teachers and school health nurses.
Method: Forty second-year (8th grade) middle school students (20 boys and 20 girls) were each rated by 17 physical education teachers and 20 school health nurses as they were observed being pressured by peers to smoke or drink (based on both verbal and non-verbal items).
Results: Test-retest reliability showed a high reliability coefficient of 0.8 or higher for four out of five of the teachers for verbal exchange rating. Many non-verbal evaluations showed a reliability coefficient of 0.75 or lower. Two groups of physical education teachers and school health nurses were further divided into two groups based on work experience to evaluate reliability. As a result, the intra-class correlation coefficients (ICC) for verbal evaluations of refusal skills for both smoking and drinking were high when the more experienced physical education teachers were excluded. However, all four groups showed an inadequate reliability coefficient for non-verbal expressions. Additionally, it was observed that the experienced physical education teachers tended to be harsher in their rating of subjects while the experienced school health nurses tended to be more lenient.
Conclusion: The evaluation of the middle school student refusal skills was suffi ciently reliable for verbal responses, but the evaluation of non-verbal responses did not exhibit sufficient reliability.
