Reporting of reliability coefficients is an important procedure in articles describing the development of new psychological scales. However, it appears that Japanese psychology researchers have not yet arrived at a consensus regarding what constitutes a desirable magnitude for a reliability coefficient. In this study, I conducted a meta-analysis summarizing 65 test-retest correlations from 58 studies published in the Japanese Journal of Psychology, which is a highly-ranked peer-reviewed psychological journal in Japan. The results of a meta-analysis, which involved the use of a random effect model, showed that the desirable mean test-retest correlation was ρ =.76 (95% CI =.70–.81). There was no significant relationship between the test-retest correlations and coefficients alpha. The number of items of the scale correlated positively with the test-retest correlation coefficients. Researchers tended to mention problems regarding test-retest coefficients only when they were less than r =.50. The desirable usage of reliability coefficients was also discussed.
