Selective Attention-based Modulation for Continual Learning
This work addresses catastrophic forgetting in continual learning for AI systems, offering an incremental improvement through attention-based modulation.
The paper tackles the problem of catastrophic forgetting in continual learning by introducing a biologically-inspired selective attention modulation (SAM) approach, which enhances state-of-the-art methods by up to 20 percentage points in performance and improves feature robustness against spurious features and adversarial attacks.
We present SAM, a biologically-plausible selective attention-driven modulation approach to enhance classification models in a continual learning setting. Inspired by neurophysiological evidence that the primary visual cortex does not contribute to object manifold untangling for categorization and that primordial attention biases are still embedded in the modern brain, we propose to employ auxiliary saliency prediction features as a modulation signal to drive and stabilize the learning of a sequence of non-i.i.d. classification tasks. Experimental results confirm that SAM effectively enhances the performance (in some cases up to about twenty percent points) of state-of-the-art continual learning methods, both in class-incremental and task-incremental settings. Moreover, we show that attention-based modulation successfully encourages the learning of features that are more robust to the presence of spurious features and to adversarial attacks than baseline methods. Code is available at: https://github.com/perceivelab/SAM.