An Effective Constructive Algorithm of Single Decision Tree Preserving Predictive Performance of Ensemble Learning

Abstract

Decision tree is an useful model for label classification and has high interpretability. However, the common size of training data prepared for a decision tree could lead to overfitting. Although the ensemble discriminator of decision tree prevents overfitting and earns high predictive accuracy, it will lose interpretability because of generating a large amount of random decision trees. Therefore, if we can learn a single decision tree that has the same predictive performance to the ensemble discriminator, it should be useful for actual application. In this study, we propose a method for learning a single decision tree with high accuracy with Autoencoder as a generative model, and we also use SMOTE as oversampling method to generate additional learning data by following the distribution of the target data with a small amount of computation. Finally, we show the effectiveness of the proposed method by actual data.