Study towards the proposal of a new Zero-shot NN evaluation index

Abstract

Neural Architecture Search (NAS), which automatically optimizes the structure of neural networks, garner attention in recent years. NAS has the problem that it takes an enormous amount of time to search. For this reason, a zero-shot evaluation method has been proposed to estimate classification accuracy without training. The previously proposed zero-shot indices assess performance by defining expression based on the activity of the output or the derivative of the output with respect to the input. However, these indices tend to overestimate the performance of Neural Networks with a high number of parameters. We therefore decid to investigate whether there is a way to solve this problem. We observe that the robustness of the ReLU output distribution with respect to the weights increases when the performance of the neural network is high.