Fourier Residual Networks Achieve Spectral Accuracy for Discontinuous Functions
Provides a theoretical framework for spectral convergence in neural networks for a broad class of one-dimensional functions, including those with jump discontinuities.
Fourier residual networks achieve spectral accuracy for discontinuous functions without requiring periodicity or continuity, overcoming limitations of classical Fourier methods and nonlinear approaches.
We present a constructive approximation framework for analyzing the expressive power of Fourier residual networks in approximating a broad class of one-dimensional functions. Our study covers both piecewise continuous functions -- including those with jump discontinuities in the function and its derivatives -- and fully smooth functions. We show that Fourier residual networks achieve spectral convergence without requiring periodicity or continuity, thereby overcoming key limitations of classical linear Fourier approximation and nonlinear methods, without being restricted to Barron-type function spaces. Our approach builds on classical techniques from approximation theory, including fixed-point iteration and Hermite interpolation by trigonometric polynomials. We support our theoretical results with numerical experiments based on both the constructed approximations and a randomized algorithm developed in our earlier work.