Scientific Epistemology of Evidence-Based Policy Making and its Implications for Higher Education Research

Abstract

　 Evidence-based policy making (EBPM) has become an important research issue in the research on higher education. However, skepticism continues about randomized controlled trials (RCTs), which is the main methodology of EBPM. In addition, recent EBPM research has raised epistemological doubts; the evidence built by RCTs lacked the evidential capacity to illustrate the effectiveness of policies. Therefore, this study intends to elucidate the characteristics of evidence required by EBPM and its construction by conducting a theoretical study from the perspective of scientific epistemology.

　 This study focuses on EBPM theory, that is, the theory of evidence for use, which advocates a research design that ensures external validity. RCTs are excellent in terms of internal validity but exhibit limitations in terms of external validity. The objective of this theory is to construct usable evidence for the effective prediction of policies. Behind these arguments, we find insights into scientific epistemology, especially the contemporary scientific realism debate.

　 This process clarifies that insight into scientific realism debate can be applied to the natural sciences and the social sciences, especially to higher education research. Moreover, we present two propositions. The first is that EBPM requires a hierarchically structured theory, is composed of concepts at varying levels of abstraction, and requires empirical evidence across contexts combined with the hierarchically structured theory. The second is that testing the validity of the theory in contexts that were not envisioned during its development is necessary.

　 Based on these findings, this study concludes with a discussion of the challenges in developing EBPM in higher education research and the solutions for such challenges.