Neural Network Quantum Field Theory from Transformer Architectures
This work provides a novel method for connecting neural networks to quantum field theory, which could impact theoretical physics and machine learning, though it appears incremental as it builds on existing NN-QFT frameworks.
The authors tackled the problem of constructing Euclidean scalar quantum field theories using neural networks, specifically transformer attention heads, and found that non-Gaussian field statistics persist in the infinite-width limit, but summing many heads suppresses non-Gaussian correlators as 1/N_h, yielding a Gaussian theory in the large-head limit.
We propose a neural-network construction of Euclidean scalar quantum field theories from transformer attention heads, defining $n$-point correlators by averaging over random network parameters in the NN-QFT framework. For a single attention head, shared random softmax weights couple different width coordinates and induce non-Gaussian field statistics that persist in the infinite-width limit $d_k\to\infty$. We compute the two-point function in an attention-weight representation and show how Euclidean-invariant kernels can be engineered via random-feature token embeddings. We then analyze the connected four-point function and identify an "independence-breaking" contribution, expressible as a covariance over query-key weights, which remains finite at infinite width. Finally, we show that summing many independent heads with standard $1/N_h$ normalization suppresses connected non-Gaussian correlators as $1/N_h$, yielding a Gaussian NN-QFT in the large-head limit.