Robust Fourier Neural Networks
This work addresses noise robustness in Fourier neural networks, which is an incremental improvement for applications in signal processing or data with noisy measurements.
The paper tackles the problem of high generalization errors in Fourier neural networks when labels or measurements are noisy, and shows that adding a diagonal layer after Fourier embedding improves robustness to noise, enabling learning of sparse Fourier features.
Fourier embedding has shown great promise in removing spectral bias during neural network training. However, it can still suffer from high generalization errors, especially when the labels or measurements are noisy. We demonstrate that introducing a simple diagonal layer after the Fourier embedding layer makes the network more robust to measurement noise, effectively prompting it to learn sparse Fourier features. We provide theoretical justifications for this Fourier feature learning, leveraging recent developments in diagonal networks and implicit regularization in neural networks. Under certain conditions, our proposed approach can also learn functions that are noisy mixtures of nonlinear functions of Fourier features. Numerical experiments validate the effectiveness of our proposed architecture, supporting our theory.