Hyperparameter Optimization for Convolutional Neural Networks by Evolutionary Multi-objective Optimization with Multiple Datasets

Abstract

Convolutional Neural Networks (CNNs) achieve high classification performance in object recognition. Although CNNs automatically learn their network weights by data-driven methods, the appropriate hyperparameters of the network structure must be selected to achieve high classification performance. In general, the appropriate hyperparameters depend on the dataset to be used. In addition, because finding the optimal hyperparameters by manual adjustment is difficult, automatic adjustment using search methods has been actively studied. In this research, we optimize the number of filters, which is one of the hyperparameters in CNNs, by using evolutionary multi-objective optimization to enhance the generalization ability. Specifically, we use multiple datasets and define multiple objective functions based on the loss functions for those datasets. That is, the number of objectives is the same as the number of datasets. In computational experiments, we compare the classification accuracy of CNNs optimized by the multi-objective optimization method and the single-objective optimization method.