Functional Analytical Methods for Neural Networks and the Infinite-Dimensional Null Space

Abstract

In this paper, we present recent results on the integral representation of neural networks. In the theoretical study of deep learning, using the integral representation to treat neural nets in a functional analytic manner is developing. However, the structure of the domain space of the integral representation operator S is mostly unknown. It is less known that there even exists an infinite-dimensional null space ker S. In this paper, we consider several problems related to integral representations and discuss the characterizations of and ker S in each context.