Unlocking the Potential of Similarity Matching: Scalability, Supervision and Pre-training
This work addresses the need for more efficient and biologically plausible learning algorithms in machine learning, though it is incremental in combining existing methods.
The study tackled the limitations of backpropagation by scaling the biologically plausible similarity matching framework to large datasets, achieving competitive performance with backpropagation-trained models in feature evaluation.
While effective, the backpropagation (BP) algorithm exhibits limitations in terms of biological plausibility, computational cost, and suitability for online learning. As a result, there has been a growing interest in developing alternative biologically plausible learning approaches that rely on local learning rules. This study focuses on the primarily unsupervised similarity matching (SM) framework, which aligns with observed mechanisms in biological systems and offers online, localized, and biologically plausible algorithms. i) To scale SM to large datasets, we propose an implementation of Convolutional Nonnegative SM using PyTorch. ii) We introduce a localized supervised SM objective reminiscent of canonical correlation analysis, facilitating stacking SM layers. iii) We leverage the PyTorch implementation for pre-training architectures such as LeNet and compare the evaluation of features against BP-trained models. This work combines biologically plausible algorithms with computational efficiency opening multiple avenues for further explorations.