A singular Riemannian Geometry Approach to Deep Neural Networks III. Piecewise Differentiable Layers and Random Walks on $n$-dimensional Classes
This work provides a theoretical tool for analyzing neural networks in non-Euclidean settings, which is incremental as it builds on prior geometric approaches.
The authors tackled the problem of understanding deep neural networks by extending a singular Riemannian geometry framework to convolutional, residual, and recursive networks, including non-differentiable activations like ReLU, and demonstrated results through numerical experiments on image classification and thermodynamic problems.
Neural networks are playing a crucial role in everyday life, with the most modern generative models able to achieve impressive results. Nonetheless, their functioning is still not very clear, and several strategies have been adopted to study how and why these model reach their outputs. A common approach is to consider the data in an Euclidean settings: recent years has witnessed instead a shift from this paradigm, moving thus to more general framework, namely Riemannian Geometry. Two recent works introduced a geometric framework to study neural networks making use of singular Riemannian metrics. In this paper we extend these results to convolutional, residual and recursive neural networks, studying also the case of non-differentiable activation functions, such as ReLU. We illustrate our findings with some numerical experiments on classification of images and thermodynamic problems.