Purpose: To assess problems of a system for evaluating educational methods which cause interevaluator variability at the pulse and blood pressure measurement station of the objective structured clinical examination. Subjects: 186 evaluations for 93 fourth-year medical students. Method: The vital-sign station consists of pulse examination, blood pressure measurement, and a 1-minute oral examination. To assess interevaluator reliability, the differences between two evaluations of each of 15 evaluation steps were calculated and divided into three categories: “greater than 95% agreement, ” “unidirectional disagreement, ” and “bidirectional disagreement.” Results: The steps of “consent to examination, ” “proper verbal instructions, ” “valve release, ” “estimation by palpation”, a question about “normal systolic blood pressure, ” and a question about “the interval and the number of repetitions” showed greater than 95% agreement. The steps of “manner of speaking, ” “palpation of peripheral pulse, ” “stethoscope placement, ” “cuff deflation, ” “presentation of blood pressure, ” and a question about “absolute arrhythmia” showed unidirectional disagreement. The steps of “presentation of pulse measurement, ” “cuff wrapping, ” and “cuff inflation” showed bidirectional disagreement. Discussion: Suggestions to improve intervaluator reliability include:(1) decreasing the evaluation steps to two with a single checkpoint, (2) presenting blood pressure with the palpation method, (3) deleting the oral examination, (4) providing adequate instruction about the differences in the types of cuffs and bladders, and (5) clarifying evaluation criteria and the training of evaluators.
View full abstract