Anatomy of Deep Neural Networks―A Statistical Mechanics Approach

Abstract

Statistical mechanics of a deep neural network, feedforward network of perceptrons, is studied using the replica method. We considered two scenarios 1) random scenario 2) teacher-student scenario in a Bayes optimal setting. The analysis performed in the thermodynamics limit revealed characteristic wetting transitions in the solution space.