Understanding Deep Learning using Topological Dynamical Systems, Index Theory, and Homology
This work provides a novel mathematical framework for understanding deep learning, which could benefit researchers in AI and mathematics by offering new insights into model behavior, though it is incremental in applying existing topological tools to a new domain.
The paper tackles the challenge of interpreting deep learning models by applying topological dynamical systems, index theory, and homology to analyze neural networks, showing that trained networks on MNIST have sinks and basins of attraction for classes, but these lie in regions not corresponding to actual digit images, with index theory implying the existence of saddles.
In this paper we investigate Deep Learning Models using topological dynamical systems, index theory, and computational homology. These mathematical machinery was invented initially by Henri Poincare around 1900 and developed over time to understand shapes and dynamical systems whose structure and behavior is too complicated to solve for analytically but can be understood via global relationships. In particular, we show how individual neurons in a neural network can correspond to simplexes in a simplicial complex manifold approximation to the decision surface learned by the NN, and how these simplexes can be used to compute topological invariants from algebraic topology for the decision manifold with an explicit computation of homology groups by hand in a simple case. We also show how the gradient of the probability density function learned by the NN creates a dynamical system, which can be analyzed by a myriad of topological tools such as Conley Index Theory, Morse Theory, and Stable Manifolds. We solve analytically for associated the differential equation for a trained NN with a single hidden layer of 256 Neurons applied to the MINST digit dataset, and approximately numerically that it a sink and basin of attraction for each of the 10 classes, but the sinks and strong attracting manifolds lie in regions not corresponding to images of actual digits. Index theory implies the existence of saddles. Level sets of the probability functions are 783-dimensional manifolds which can only change topology at critical points of the dynamical system, and these changes in topology can be investigated with Morse Theory.