Theory of Realistic Evaluation: Implications for Evidence-Based Policy Making in Japan

Abstract

  One of the challenges of Evidence-Based Policy Making (EBPM) in Japan is to ensure the external validity of evidence. We focus on the theory of realistic evaluation to analyze this aspect in our study. The theory’s main feature is its emphasis on theory building rather than on data analysis while constructing and utilizing evidence. Although realistic evaluation methodology has attracted considerable attention in domestic and international evaluation research, how the epistemological premises are reflected in the evaluation theory remains ambiguous. This has hindered the understanding and development of the theory and its practical application. Therefore, the focus of this study is the epistemological dimension of realistic evaluation to clarify the core idea of the theory. We analyzed the significance and the limitations of the theory of realistic evaluation as a methodology for program evaluation. Furthermore, from perspective of realistic evaluation, we criticize the methodology of experimental and quasi-experimental methods. In conclusion, the promotion of EBPM requires a greater emphasis on constructing a middle-range theory with high applicability than attaining evaluation results with high external validity.