Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines
This work addresses the problem of learning in systems where exact gradients are impractical, such as brains and neuromorphic hardware, though it appears incremental as an alternative to existing methods.
The paper tackles the challenge of implementing gradient-based learning in biological and neuromorphic systems by introducing Ornstein-Uhlenbeck adaptation (OUA), a noise-driven method that balances exploration and exploitation using global reinforcement signals, and demonstrates its viability across tasks like supervised learning, reinforcement learning, and meta-learning.
Learning is a fundamental property of intelligent systems, observed across biological organisms and engineered systems. While modern intelligent systems typically rely on gradient descent for learning, the need for exact gradients and complex information flow makes its implementation in biological and neuromorphic systems challenging. This has motivated the exploration of alternative learning mechanisms that can operate locally and do not rely on exact gradients. In this work, we introduce a novel approach that leverages noise in the parameters of the system and global reinforcement signals. Using an Ornstein-Uhlenbeck process with adaptive dynamics, our method balances exploration and exploitation during learning, driven by deviations from error predictions, akin to reward prediction error. Operating in continuous time, Orstein-Uhlenbeck adaptation (OUA) is proposed as a general mechanism for learning dynamic, time-evolving environments. We validate our approach across diverse tasks, including supervised learning and reinforcement learning in feedforward and recurrent systems. Additionally, we demonstrate that it can perform meta-learning, adjusting hyper-parameters autonomously. Our results indicate that OUA provides a viable alternative to traditional gradient-based methods, with potential applications in neuromorphic computing. It also hints at a possible mechanism for noise-driven learning in the brain, where stochastic neurotransmitter release may guide synaptic adjustments.