Characterization of the typical deep learning solutions is crucial to understanding and controlling deep learning. Due to the complex structure of real deep neural networks (NNs), various simplified mathematical models are employed in conventional theoretical analysis. In this study, we describe a mathematical model of a single hidden layer in an NN, which is an integral representation of NNs, and its right inverse operator (or analysis operator), the ridgelet transform. Furthermore, while the classical ridgelet transform was obtained heuristically, we had recently developed a natural technique to derive it. As an application, we succeeded in developing an NN on manifolds (noncompact symmetric spaces) and deriving the associated ridgelet transform.

Optimization tools have been extensively employed in signal processing. Recent advances in optimization algorithms based on proximity operators have broadened the application range of optimization-based signal processing. Moreover, deep learning has rapidly developed a new signal processing scheme that can perform notably better than conventional ones. Consequently, they have been combined in some studies. In this paper, we briefly review studies combining proximal splitting algorithms and deep learning.