Sparse Regularization Method in Marginal Regression Models

Abstract

Sparse regularization methods, such as Lasso, now have become a kind of standard ones when the number of covariates is comparative to the sample size. Meanwhile, due to the estimation accuracy and the computational burden, it would often be effective to screen important variables in advance rather than to apply such method in any case, if the number of covariates are exponentially larger than the sample size. In this article, we propose a screening method in ultra high-dimensional scenario. We first explain a covariance based screening method is equivalent to some sparse regularization method for marginal regression models with Lasso-type penalty. We also show the such screening methods enjoys the sure screening property, that is, the selected model includes underlying target model with high probability. In order to choose tuning parameter without any arbitrarily, we consider the choice based on multiple testing and explain the selected model would not be much large with high probability. Furthermore, we compare the performance of several screening methods through simulation studies and apply a covariance based screening method to leukemia dataset.