Although null hypothesis significance testing has been strongly criticized for decades, it has been the dominant statistical method in the field of psychology. Non-reproducibility of findings in psychology can be attributed, at least partially, to an arbitrary threshold (i.e., .05) in the null hypothesis significance testing and overrepresentation of p-values. The present study surveyed papers from the Japanese Journal of Social Psychology and examined whether or not such overrepresentation also existed among psychology researchers in Japan. Effect size measures and p-values did not correspond well when p-values were set at around .05. Moreover, the frequency of p-values just below .05 was greater than expected. These results imply that the overrepresentation of p-values can produce unreliable and irreproducible results. Two types of remedies are discussed to alleviate the problems of overrepresentation of the p-values.
