Experimental Investigation on Adaptability of SE Module to DenseNet

Abstract

Recently, layer stack approach for CNN (Convolutional Neural Network) has achieved high image recognition performance. However, as the number of stacked layers is increased, this leads to increasing number of parameters. Therefore, a high-speck machine is required for calculation. To solve this problem, in this research, we focus on SE module, which has an attention mechanism that adaptively trains the relationship among filters and achieves the improvement of recognition performance with a slight increase in the number of parameters. Although this module has achieved high parameter efficiency compared with the layer stack approach, the adaptability of the SE module is discussed insufficiently. Therefore, in this paper, we evaluate its adaptability by utilizing DenseNet and ResDenseNet, which have higher parameter efficiency compared with ResNet and both have DenseBlock modules. Unfortunately, a simple combination of SE module and such CNNs generally adds the SE module to end of each CNN module, which leads increasing a large number of parameters since the SE module requires parameters depending on the number of filters in DenseBlock. To solve this problem, we propose a new combination of SE and DenseBlock modules, that is, we add the SE module to each branching function. From the empirical evaluation with CIFAR10 and CIFAR100, our proposed method improved recognition performance compared with DenseNet/ResDenseNet without SE modules.