The Reproducibility Crisis in Science Education Studies: Causes and Solutions

Abstract

In recent years, the reproducibility crisis has become a problem in many disciplines, including education, in which the results of past research have not been reproduced. This crisis is caused by questionable research practices (QRPs). The purpose of this study is to clarify the existence of QRPs in the field of science education in Japan and to propose a countermeasure to the reproducibility problem. We analyzed the QRPs in articles published in the Journal of Research in Science Education in the past 4 years. As a result, eight types of QRPs (lack of validation, biased sampling, publication bias, false multiple comparisons, low statistical power, HARKing, over-generalization, and insufficient information) were found to have been used in the articles. Finally, we proposed four ideas for actions that researchers, practitioners, and editorial boards should take to prevent QRPs and solve the reproducibility problem: conducting replication studies, disseminating appropriate research methods, introducing a pre-registration system, and promoting open science.