Neural Network Field Theories: Non-Gaussianity, Actions, and Locality
This work bridges theoretical physics and machine learning, offering new tools for analyzing neural networks and designing architectures, though it is incremental in extending known connections.
The paper tackles the correspondence between neural network ensembles and field theories, showing that expansions beyond the infinite-width limit can lead to interacting field theories with advantages like improved universal approximation. It presents a method to reconstruct actions from correlators and demonstrates this by realizing φ⁴ theory as an infinite-width neural network field theory.
Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. When the central limit theorem can be applied in the infinite-width (infinite-$N$) limit, the ensemble of networks corresponds to a free field theory. Although an expansion in $1/N$ corresponds to interactions in the field theory, others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories. These other expansions can be advantageous over the $1/N$-expansion, for example by improved behavior with respect to the universal approximation theorem. Given the connected correlators of a field theory, one can systematically reconstruct the action order-by-order in the expansion parameter, using a new Feynman diagram prescription whose vertices are the connected correlators. This method is motivated by the Edgeworth expansion and allows one to derive actions for neural network field theories. Conversely, the correspondence allows one to engineer architectures realizing a given field theory by representing action deformations as deformations of neural network parameter densities. As an example, $φ^4$ theory is realized as an infinite-$N$ neural network field theory.