Predictive Principal Variable Selection Criteria for Multivariate Multiple Regression Model with an Application

Abstract

　Multivariate multiple linear regression analysis is one of the powerful statistical tools to simultaneously predict two or more response variables through two or more explanatory variables. However, similar to multiple linear regression analysis with one response variable, multivariate multiple linear regression analysis often suffers from the multicollinearity problem. To solve the problem, we focus on the fact that the variance-covariance matrix of the estimated regression coefficients is expressed as the Kronecker product of two matrices: the residual variance matrix of the response variables and the inverse of the sum-of-squares matrix of explanatory variables. Then, we construct four types of variable selection criteria based on a simple idea that selects a subset of explanatory variables using the variance-covariance matrix of the estimated regression coefficients. In addition, we propose both the full-search variable selection methods and the stepwise variable selection methods. Through the numerical experiments and an application to the quality control, we show that the proposed variable selection methods enable the selection of a subset of explanatory variables that fit the response variables well while avoiding the multicollinearity of explanatory variables.