Neural Oscillators are Universal
This work offers a foundational theoretical justification for using neural oscillators in machine learning, which could impact sequence modeling, graph learning, and analog devices, though it is incremental in building on existing oscillator-based systems.
The authors tackled the problem of establishing a theoretical foundation for neural oscillators in machine learning by proving that they are universal approximators of continuous and causal operators between time-varying functions, providing justification for their use in various architectures.
Coupled oscillators are being increasingly used as the basis of machine learning (ML) architectures, for instance in sequence modeling, graph representation learning and in physical neural networks that are used in analog ML devices. We introduce an abstract class of neural oscillators that encompasses these architectures and prove that neural oscillators are universal, i.e, they can approximate any continuous and casual operator mapping between time-varying functions, to desired accuracy. This universality result provides theoretical justification for the use of oscillator based ML systems. The proof builds on a fundamental result of independent interest, which shows that a combination of forced harmonic oscillators with a nonlinear read-out suffices to approximate the underlying operators.