The Effect of Feedback on Quantitative Halo Rater Error in Rater Training : The Estimation of Halo Error Using Second-order Factor Analysis (Part 2)(Theory and Methodology)

Abstract

It is important to know whether or not performance ratings provide an accurate reflection of an individual's performance. Despite the accuracy of performance ratings, it is generally acknowledged that they are too often contaminated by systematic halo rater errors. Rater training is an area that has shown some promise for improving the effectiveness of performance ratings. The purpose of this study is to empirically examine the effectiveness of rater training measured by halo rater errors as psychometric outcomes for rating and the effect of reducing errors in the rater judgment process by providing feedback for two items: (1) quantitative errors of each rater with factor scores that are estimated by a confirmatory factor analysis model, and (2) whether or not the rater committed errors. The results of the experiment attended by a total of 116 business persons clearly show that the number of raters who committed halo rater errors the first time reduced was lower the second time as the result of the abovementioned feedback. This suggests two things; rater training is effective for reducing halo rater errors, and the training process is effective for reducing halo rater errors.