Study of model evaluation method and of selection method of tuning parameter in Lasso

Abstract

High-dimensional data sets have been used extensively in recent years. However, their use is computationally expensive, and model features are difficult to capture. Thus, it is necessary to construct a simple and appropriate model. The least absolute shrinkage and selection operator (Lasso) can estimate several regression coefficients as exactly zero, making it possible to estimate a model and select variables at the same time. However, several problems exist. The decision procedure for the tuning parameter, which is the coefficient of the penalty term, is not established, and it is difficult to determine the value of the estimated model.

In this paper, we consider a model evaluation method and a selection method for the tuning parameter. We investigate selection methods for the most appropriate tuning parameter and model evaluation methods using the model evaluation indexes RSS, AIC, BIC, and Mallows’s Cp. Let several regression coefficients be zero and vary the number of zeroes. Then, we observe the performances of the evaluation indexes. Furthermore, we observe their performances when the values of the regression coefficients are close to zero and confirm them by simulation.