Visualizing the Behavior of the Inner Layers of Convolutional Neural Networks by Layer-wise Relevance Propagation

Abstract

In recent years, complex machine learning models like deep neural networks play a central role in many real applications, due to their high predictive performance. Interpreting machine learning models is then considered to be important since practitioners constantly need clues for improvement on such complex models, whose behavior is not directly visible to human. In this paper, we focus on the inner workings of convolutional neural networks, visualize them by a method called layer-wise relevance propagation, and report several findings from the visualization.