Variable Section in Linear Regression based on Continuous Minimization of Information Criterion

Abstract

We consider the task of variable selection in linear regression models. Because of their simplicity, linear regression models are widely used for prediction and forecasting. When applying linear regression models, it is important to conduct variable selection, for which we simultaneously select a subset of relevant input variables and optimize model parameters. By applying the SICM (Sequential Information Criterion Minimization) algorithm, which was proposed in our previous work, we propose a solution of the task. The algorithm enables us to continuously minimize an information criterion, which includes L0 norm such as the number of parameters, and was applied to logistic regression models and their mixtures. In this paper, we derive a method for continuously minimizing an information criterion in linear regression models.