Limits of spectral learning under noise
For researchers using spectral methods in scientific inference, this work identifies intrinsic limits on recovering functional structure from noisy data, providing a theoretical understanding of noise-induced instability.
The paper studies supervised regression with additive label noise using sparse spectral representations, deriving a closed-form expression for the overlap between noisy and noiseless coefficient vectors that reveals a universal degradation curve governed by a single intrinsic noise scale. Numerical experiments across multiple bases confirm the theoretical prediction, showing a fundamental noise threshold beyond which coefficient estimates become unstable.
Learning functional relationships from noisy data is a central problem in scientific inference. Spectral methods approximate unknown functions by expanding them in a basis and estimating the corresponding coefficients from data, but the stability of these coefficients under noise remains poorly understood. Here we study supervised regression with additive label noise using sparse spectral representations across multiple bases and dimensions. We show that noise induces a predictable drift in the learned coefficient vector whose magnitude depends on the effective number of active spectral modes. After whitening the empirical feature geometry, we derive a closed-form expression for the overlap between noisy and noiseless coefficient vectors, revealing a universal degradation curve governed by a single intrinsic noise scale. Numerical experiments across Fourier, Legendre, Bessel, and Haar bases confirm the theoretical prediction. The results demonstrate that spectral learning exhibits a fundamental noise threshold beyond which coefficient estimates become unstable, placing intrinsic limits on recovering functional structure from noisy data.