Case studies of “p-hacking” in social psychology

Abstract

There is currently an ongoing debate about reproducibility in social psychology. One reason for low reproducibility is the excessive use of questionable research practices, called “p-hacking”. We present two direct replication studies of social priming and embodied cognition that failed to replicate the original findings under the circumstances of high statistical power. However, a variety of p-hacking attempts made it possible to obtain some false-positive findings based on the data from these two studies. We note that selectively reporting the results and deriving the hypothesis after the results are obtained may disguise the presence of p-hacking, and argue that pre-registration of studies and fair publishing of negative results could inhibit p-hacking.