GAによるパレート最適な決定木集合の生成

抄録

The construction of decision trees by the ID3 algorithm is a very well known approach to inductive learning. However, it is necessary to prune decision trees constructed by the ID3 algorithm, because it cannot deal well with uncertainty due to noise in the data. When redundant features are included in a given feature set, the ID3 often tend to generate over-specialized decision trees. In recent years, the feature selection problem has been closed up in machine learning. This paper emphasizes that the feature selection as pre-processing, the complete tree generation as central-processing, and the pruning as post-processing should be unified. In general, accuracy and simplicity are requested of a decision tree. There is a tradeoff relation between accuracy and simplicity. This paper emphasizes that the decision tree induction should be formulated as a multi-objective optimization problem. The rational solutions of such a problem are known as Pareto optimal. This paper presents a genetic algorithm for generating Pareto optimal decision trees at once. The fitness is defined as a vector function of minimizing the error rate and minimizing the number of leaf nodes. The sub-trees exchange crossover and a sub-tree insertion as mutation are adopted to generate new decision trees. The non-Pareto optimal selection strategy is introduced as a model of generation alternation. Under this strategy, the population can come near to the true Pareto optimal set in progression. The algorithm is applied to the Digit benchmark problem and compared with the traditional approaches. The experiments show that the proposed algorithm can generate Pareto optimal solutions that dominate completely solutions obtained by the existing methods.