Problems in the Evaluation of Medical Interviewing Skills with Objective Structured Clinical Examinations:

How Can Reasonable Objectivity Be Ensured?

Abstract

Students' interviewing skills are now commonly evaluated with standardized patient-based assessment methods. Four pairs of instructors at Nihon University School of Medicine used objective structured clinical examinations to evaluate the medical interviewing skills of 122 fifth-year medical students. The results were then analyzed to improve the accuracy of rating with objective structured clinical examinations. Interrater variability was significant among the two pairs of instructors. Variability was greatest when instructors evaluated a student's performance but was minimal when they judged whether a student had carried out a task. The number of standardized patients was 8, with the average score of each standardized patient ranging from 52.5 to 73.3 (full score, 100). These results suggest that the rating process for each item should be further refined and that the standard for evaluation should be clarified.