Information-Theoretic Principal Variable Selection Criterion and Its Properties

Abstract

In order to reduce a large number of original variables to a smaller one that still contains most of the correlation information in statistical data without an external criterion, we propose ``the information-theoretic principal variable (iPV) selection criterion", from the perspective of Kullback—Leibler divergence. In addition, from the viewpoint of statistical parameters, we show that (a) the iPV selection criterion has a stopping rule, and (b) if a set of non-selected variables by the stopping rule is characterized as the independent set then the value of the iPV selection criteria is zero.