Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning
This work addresses the need for adaptable spectral bases in scientific machine learning, offering a novel architecture for handling both periodic and nonperiodic functions, though it appears incremental in the context of Fourier-inspired neural networks.
The authors tackled the problem of representing multidimensional nonharmonic Fourier series in neural networks by introducing the Fourier Learning Machine (FLM), which uses a feedforward structure with cosine activations to learn frequencies, amplitudes, and phase shifts as trainable parameters, achieving performance comparable or superior to established architectures like SIREN and vanilla feedforward NNs on benchmark PDEs and optimal control problems.
We introduce the Fourier Learning Machine (FLM), a neural network (NN) architecture designed to represent a multidimensional nonharmonic Fourier series. The FLM uses a simple feedforward structure with cosine activation functions to learn the frequencies, amplitudes, and phase shifts of the series as trainable parameters. This design allows the model to create a problem-specific spectral basis adaptable to both periodic and nonperiodic functions. Unlike previous Fourier-inspired NN models, the FLM is the first architecture able to represent a complete, separable Fourier basis in multiple dimensions using a standard Multilayer Perceptron-like architecture. A one-to-one correspondence between the Fourier coefficients and amplitudes and phase-shifts is demonstrated, allowing for the translation between a full, separable basis form and the cosine phase--shifted one. Additionally, we evaluate the performance of FLMs on several scientific computing problems, including benchmark Partial Differential Equations (PDEs) and a family of Optimal Control Problems (OCPs). Computational experiments show that the performance of FLMs is comparable, and often superior, to that of established architectures like SIREN and vanilla feedforward NNs.