Introduction : Meta-analysis is well recognized as the most important study methodology in pharmacoepidemiology. The cause of heterogeneity of the effects among studies in the conventional meta-analyses, has been typically analyzed by meta-regression and sometimes by extracting several studies in a post hoc manner, constructing subgroups from these studies and analyzing the effect in this subgroup. However, if multiple study subgroups are produced in a post hoc manner, since the potential possible number of subgroups is very huge, the multiplicity of testing results in the inflation of the type I error rate. Therefore, even when a significant subgroup has been identified, it can represent a type I error, due to multiplicity of testing. To insist on the significance of a post hoc subgroup analysis, it is indispensable to conduct an analysis adjusted for multiplicity.
Objective : The present study was undertaken to establish a method for resolving the problem for the multiplicity of subgroup analysis in meta-analysis.
Methods : Performance comparisons among the Bonferroni method, the Holm method, the Scheffe type method and the closed testing procedure were conducted, assuming the actual meta-analysis of clinical studies on colon cancer.
Results : In the subgroup analysis without adjustment for multiplicity, the probability of type I error was unacceptably high. On the other hand, the four methods mentioned above can control this probability to below the nominal significance level. Under many situations, the closed testing procedure showed a relatively higher power, and this method was particularly superior to the other methods when a relatively high percentage of studies revealed minor effects.

View full abstract