An efficient test method for noise robustness of deep neural networks

Abstract

A pattern recognition system is trained by using a training data set composed of input data and corresponding desired output data. After the training, the performance of the system is evaluated from certain perspectives. One is the misclassification rate (MCR) for a test data set, which is a data set separated from the training data set used in the training. The strength against noise, i.e., the noise robustness, is also an important performance measure. The noise robustness of a system is estimated by testing the MCR for a data set in which the inputs are corrupted by artificial noise. However, this test procedure can be computationally expensive, because a large number of corrupt inputs have to be created in order to cover the variability of the noise and the classification procedure has to be run for all of them. In this paper, based on a perturbative approximation method, we derive an effective test method for the noise robustness of pattern recognition systems based on deep neural networks. We demonstrate the validity of our method through numerical experiments using the MNIST data set and show that our method is much faster than the conventional expensive test method.