Development of a new criterion for evaluating the predictive ability of nonlinear regression models without cross-validation

Abstract

Many kinds of nonlinear regression methods have been developed to construct predictive models even in the existence of the nonlinear relationship between objective variables and explanatory variables. However, even when very accurate regression models are constructed, the constructed models exhibit poor predictive performance for new data. Therefore, these regression models must be validated to quantify their predictive ability and allow the appropriate model and hyperparameters to be selected. We propose predictive performance criteria for nonlinear regression models without cross-validation. The proposed criteria are the determination coefficient and the root-mean-square error for the midpoints between k-nearest-neighbor data points. These criteria can be used to evaluate predictive ability after the regression models are updated, whereas cross-validation cannot be performed in such a situation. The proposed method is effective and helpful in handling big data when cross-validation cannot be applied. We analyze numerical simulation data, aqueous solubility data and toxicity data, and confirm that the proposed criteria enable the predictive ability of the nonlinear regression models to be appropriately quantified.