Flexible Phase Dynamics for Bio-Plausible Contrastive Learning
This work addresses the challenge of implementing contrastive learning in biological and neuromorphic neural networks, providing theoretical foundations for more flexible and physically realizable learning methods, though it is incremental as it builds on recent work to relax existing constraints.
The study tackled the problem of rigid and non-local learning dynamics in contrastive learning algorithms, which limit their applicability to biological and neuromorphic systems, by showing that these algorithms can be made temporally local and function with relaxed dynamical requirements, as supported by general theorems and numerical experiments across several models.
Many learning algorithms used as normative models in neuroscience or as candidate approaches for learning on neuromorphic chips learn by contrasting one set of network states with another. These Contrastive Learning (CL) algorithms are traditionally implemented with rigid, temporally non-local, and periodic learning dynamics that could limit the range of physical systems capable of harnessing CL. In this study, we build on recent work exploring how CL might be implemented by biological or neurmorphic systems and show that this form of learning can be made temporally local, and can still function even if many of the dynamical requirements of standard training procedures are relaxed. Thanks to a set of general theorems corroborated by numerical experiments across several CL models, our results provide theoretical foundations for the study and development of CL methods for biological and neuromorphic neural networks.