2024 Volume 39 Issue 3 Pages 220-227
In this study, using data from evaluations of 20 companies by seven raters in a corporate award on the theme of human resource management, I examined the consistency of company evaluations by a small number of experts in different fields and the criterion-related validity of the mean evaluations per company. The evaluation was conducted in the order of documents and interviews, and each was rated on five items. The intraclass correlations showed that the evaluations of “philosophy,” “innovation,” “leadership,” and “motivation” were consistent across raters and that the interview evaluations were more consistent than the document evaluations. Regarding “personnel” evaluations, significant consistency was found only for document evaluations. The correlations between the mean evaluations and indices of application documents showed that companies with more text in their application documents tended to be evaluated more highly. It was confirmed that the few experts’ evaluations of the companies had a moderate degree of consistency and validity; however, it was also suggested that some combinations of evaluation items and evaluation materials may not have resulted in appropriate evaluations.
