Loop Corrections to the Training and Generalization Errors of Random Feature Models
For theorists and practitioners using random feature models, this provides a more accurate error analysis accounting for finite-width fluctuations.
The paper derives loop corrections to training and generalization errors of random feature models beyond the mean-kernel approximation, obtaining scaling laws and experimental verification.
We investigate random feature models in which neural networks sampled from a prescribed initialization ensemble are frozen and used as random features, with only the readout weights optimized. Adopting a statistical-physics viewpoint, we study the training, test, and generalization errors beyond the mean-kernel approximation. Since the predictor is a nonlinear functional of the induced random kernel, the ensemble-averaged errors depend not only on the mean kernel but also on higher-order fluctuation statistics. Within an effective field-theoretic framework, these finite-width contributions naturally appear as loop corrections. We derive the loop corrections to the training, test, and generalization errors, obtain their scaling laws, and support the theory with experimental verification.