Linear Algebra and Duality of Neural Networks
This work provides a theoretical framework for understanding neural networks, which could benefit researchers in machine learning and physics, but it appears incremental as it builds on existing mathematical concepts.
The paper tackles the problem of interpreting neural network training through linear algebra and duality principles, establishing relationships between observables and observations and analyzing training statistics from a physics perspective, with examples supporting the new concepts.
Bases, mappings, projections and metrics, natural for Neural network training, are introduced. Graph-theoretical interpretation is offered. Non-Gaussianity naturally emerges, even in relatively simple datasets. Training statistics, hierarchies and energies are analyzed, from physics point of view. Duality between observables (for example, pixels) and observations is established. Relationship between exact and numerical solutions is studied. Physics and financial mathematics interpretations of a key problem are offered. Examples support all new concepts.