The Effect of Feedback on Quantitative Rating Errors in Rater Training : An Analysis Model of Merit Rating Errors (Part 2)

Abstract

It is important to know whether or not performance ratings provide an accurate reflection of the performance of individuals being rated. Despite the accuracy of performance ratings, it is generally acknowledged that they are too often contaminated by systematic rater errors (i.e., leniency, central tendency, severity, etc.). Rater training is an area that has shown some promise in improving the effectiveness of performance ratings. The purpose of this study is to empirically examine the effectiveness of rater training measured by rating errors as psychometric outcomes in rating and the effect of reducing errors in the rater judgment process through the feedback of two things; (1) quantitative errors of each rater with factor scores that are estimated by a structural equation model created by combining two models; one being a latent curve model and the other a confirmatory factor analysis model, and (2) whether or not the rater committed errors. The results of the experiment attended by a total of 116 business persons clearly show that raters who committed rating errors the first time reduced the number of error the second time as the result of the above-mentioned feedback. This suggests two things; rater training itself is effective for reducing rating errors, and the training process is effective for reducing rater errors.