Representation Learning in a Decomposed Encoder Design for Bio-inspired Hebbian Learning
This work addresses the challenge of enhancing robustness and transparency in learned classifiers for computer vision applications, representing an incremental advancement in bio-inspired learning methods.
The paper tackled the problem of improving representation learning robustness and narrowing the performance gap between Hebbian and backpropagation-based models by proposing a modular encoder framework with invariant visual descriptors as inductive biases, achieving superior performance compared to non-decomposed encoders on image and video datasets.
Modern data-driven machine learning system designs exploit inductive biases in architectural structure, invariance and equivariance requirements, task-specific loss functions, and computational optimization tools. Previous works have illustrated that human-specified quasi-invariant filters can serve as a powerful inductive bias in the early layers of the encoder, enhancing robustness and transparency in learned classifiers. This paper explores this further within the context of representation learning with bio-inspired Hebbian learning rules. We propose a modular framework trained with a bio-inspired variant of contrastive predictive coding, comprising parallel encoders that leverage different invariant visual descriptors as inductive biases. We evaluate the representation learning capacity of our system in classification scenarios using diverse image datasets (GTSRB, STL10, CODEBRIM) and video datasets (UCF101). Our findings indicate that this form of inductive bias significantly improves the robustness of learned representations and narrows the performance gap between models using local Hebbian plasticity rules and those using backpropagation, while also achieving superior performance compared to non-decomposed encoders.