A Study on Speech Evaluation in an English Speech Contest : Judges' Scoring Tendencies and Factors Affecting Evaluation

Abstract

This paper deals With the evaluation of speech performances in an English speech contest for Japanese university students. The purpose of this study is mainly two-fold. One is to investigate the consistency or stability in the ranks given to each contestant by the two judge groups, officially-invited judges and student judges. The agreement rate in scoring within and between the two types of judges was inspected. The other is to find the underlying factors that may affect the speech evaluation by means of a factor analysis. The conditions for speakers to be highly evaluated were also examined. There Was little consistency between the two judges, and within the official judges, which resulted m lower rank-order correlation coefficients and lower inter-rater reliability. On the other hand, within the student judges results were found to be relatively stable. Concerning the affecting factors, the first factor, that most affected the overall evaluation, was vocal elements (pronunciation, fluency, voice-control, etc.) followed by contents of speech, and higher points were given to these items Apart from vocal elements, physical movement was also found to be the factors for speakers to be ranked higher.