Stepwise logistic regression is the traditional and most commonly used method for identifying biomarkers and evaluating the magnitude of their effects based on clinical data. Here, we evaluated the performance of the resampling methods leave-one-out cross-validation, 10-fold cross-validation, bootstrap, and .632+ bootstrap in terms of internal validation of prediction analysis using stepwise logistic regression. We conducted simulation studies to compare the ability of these methods to estimate prediction accuracy based on simulation settings (including statistical models) derived from two real biomarker discovery studies (Ogata et al., Leukemia Research 2012; 36: 1229–1236; Yoshimi et al., Molecular Psychiatry 2016; 21: 1504–1510). The simulation results revealed that leave-one-out cross-validation, 10-fold cross-validation, and .632+ bootstrap were comparable in terms of the root mean square error. We therefore recommend the application of these methods to similar biomarker discovery studies that involve approximately ten biomarkers with or without binary biomarkers (such as sex) and various degrees of correlation between the biomarkers.

In this paper, we give a review of recent development on gatekeeping strategies and graphical approaches. First, we give an overview of fundamental theories for multiple testing, such as partition testing and closed testing procedure. We then describe how gatekeeping strategies have been developed in the last decade to handle multiplicity issues that arise in clinical trials with hierarchically structured study objectives and how graphical approaches help to visualize such gatekeeping strategies and tailor a multiple testing procedure. Finally, we describe further availability of graphical approaches to more complex clinical trial designs such as group-sequential designs and adaptive designs.