Validation of Measuring the Training Effect Using the Rivised Keyword Method

Abstract

This study revised Hiramatsu’s (2001) key word method which is a method of measuring KIRKPATRICK model’s Level 2 (learning) in order to improve the efficiency of measurement and to clarify the evaluation criteria. The revision was made based on the findings of GOLLWITZER (1999), and the content validity was confirmed by supervisors of Education Center, a professional organization for teacher training. In order to examine the criterion-related validity, a survey was conducted at two time points on the participants in training for newly hired teachers, and valid responses were obtained from 175 participants. As a result of analysis using structural equation modeling, no significant trend in the correlation between Level 1 (reaction) and Level 2 was confirmed, but the effect size was consistent with previous studies. In addition, Level 2 was shown to have an indirect effect on changes in behavioral intention, mediated by the changes of attitude toward the behavior, subjective norm, and perceived behavioral control, providing evidence in support of the criterion-related validity. Furthermore, the adoption of web forms has improved the operability.