Bio-Inspired Mamba: Temporal Locality and Bioplausible Learning in Selective State Space Models
This work addresses challenges in biologically plausible machine learning for researchers and practitioners, offering a novel framework that could enable neuromorphic hardware implementation, though it appears incremental by combining existing methods like RTRL and STDP-like rules.
The paper tackled the problem of training spiking neural networks with temporal locality and biological plausibility by introducing Bio-Inspired Mamba (BIM), which integrates biological learning principles with the Mamba architecture, resulting in competitive performance on tasks like language modeling and speech recognition while improving energy efficiency.
This paper introduces Bio-Inspired Mamba (BIM), a novel online learning framework for selective state space models that integrates biological learning principles with the Mamba architecture. BIM combines Real-Time Recurrent Learning (RTRL) with Spike-Timing-Dependent Plasticity (STDP)-like local learning rules, addressing the challenges of temporal locality and biological plausibility in training spiking neural networks. Our approach leverages the inherent connection between backpropagation through time and STDP, offering a computationally efficient alternative that maintains the ability to capture long-range dependencies. We evaluate BIM on language modeling, speech recognition, and biomedical signal analysis tasks, demonstrating competitive performance against traditional methods while adhering to biological learning principles. Results show improved energy efficiency and potential for neuromorphic hardware implementation. BIM not only advances the field of biologically plausible machine learning but also provides insights into the mechanisms of temporal information processing in biological neural networks.