A simulation study on appropriate transformations of reliability coefficient in mixed-effects meta-analysis models

Abstract

Reliability generalization is a meta-analytic technique used to synthesize the score reliability for an instrument across many studies. The concept is relatively new and therefore the methodology for this technique is not established yet, especially the appropriate form of transformation of the reliability coefficient is not well known. In this paper, a simulation study was conducted in order to examine which transformation of alpha coefficient works best by generating a population of reliability coefficients within the framework of mixed-effects meta-analysis models. The results of six forms of transformation were compared in order to find a better transformation for reliability generalization. The results implied that either log or cube root transformations performed much better than other forms of transformations. From the variance stability viewpoint, the log transformation is more recommended since it is a variance stabilizing transformation while the cube transformation is not.