Geometric Decomposition of Feed Forward Neural Networks
This work offers a foundational geometric perspective for future research in neural networks, potentially impacting all of ML/AI, though it is incremental as it builds on existing mathematical attempts.
The authors tackled the problem of understanding feed forward neural networks as black boxes by describing an inherent geometric structure, providing a framework for improving training algorithms and computing network homology.
There have been several attempts to mathematically understand neural networks and many more from biological and computational perspectives. The field has exploded in the last decade, yet neural networks are still treated much like a black box. In this work we describe a structure that is inherent to a feed forward neural network. This will provide a framework for future work on neural networks to improve training algorithms, compute the homology of the network, and other applications. Our approach takes a more geometric point of view and is unlike other attempts to mathematically understand neural networks that rely on a functional perspective.