タスクに基づくライティング・テスト自動評価採点システム信頼性・妥当性の検証

抄録

　　This paper focuses on the reliability and validity of an automated essay-scoring system for a task-based writing test. The two rating indices (number of connectives and frequency of formulaic expressions) were replaced or modified. The indices and scores of the Accuracy and Communicability tasks were investigated by using multiple regression and the system was revised. A sample of 150 second-year high school students participated in the trial of the system and the Accuracy and Communicability values were calculated by using the resulting formulas. To estimate the degree to which the indices were collectively related to the prediction of the scores, correlation analyses were conducted. The results showed a moderately high correlation between the scores of the tasks and their indices. To validate the predictions of the system, the values were compared with the Criterion scores of the high school students. Correlations between the Accuracy and Communicability scores were significant. The composite scores of the two tasks were also significantly correlated. The results of the questionnaire, which was administered to assess the adequacy of the task scores, demonstrated that the scores were acceptable and appropriate for the students.