Spectrum-Informed Multistage Neural Networks: Multiscale Function Approximators of Machine Precision
This addresses the challenge of high-precision function approximation for scientific applications like turbulent flow, representing a strong specific gain.
The paper tackles the problem of fitting complex, multi-scale dynamical systems to very high precision in scientific machine learning, achieving double floating-point machine precision (O(10^{-16})).
Deep learning frameworks have become powerful tools for approaching scientific problems such as turbulent flow, which has wide-ranging applications. In practice, however, existing scientific machine learning approaches have difficulty fitting complex, multi-scale dynamical systems to very high precision, as required in scientific contexts. We propose using the novel multistage neural network approach with a spectrum-informed initialization to learn the residue from the previous stage, utilizing the spectral biases associated with neural networks to capture high frequency features in the residue, and successfully tackle the spectral bias of neural networks. This approach allows the neural network to fit target functions to double floating-point machine precision $O(10^{-16})$.