On the use of group structure in multiple simultaneous hypothesis testing

Abstract

In multiple simultaneous hypothesis testing (MSHT), the false discovery rate (FDR) control is vital in fields such as biology, genetics, medicine, and psychology. The idea of redoing MSHT through meta-analysis, integrating null hypotheses and data from various studies, is being explored. This approach's challenge is the impact of arbitrary choice of sets of hypotheses, and there's a lack of guidelines for appropriately using background knowledge.Our study proposes a method for MSHT tasks with inherent group structures that incorporates background knowledge. We apply Storey's method independently to each group of hypotheses. This method allows significant enhancement of detection power when the prior probability of the null hypothesis is established for each group. We demonstrate this by applying our method to an MSHT task that investigates the correlation between corporate employees' psychological assessment scores and their supervisors' evaluations, showing an increase in detection power while maintaining stable FDR control. Additionally, we discuss general guidelines for safe use of background knowledge in MSHT tasks on meta-analysis.