CBPNet: A Continual Backpropagation Prompt Network for Alleviating Plasticity Loss on Edge Devices
It addresses the problem of reduced learning ability in frozen pretrained models for edge applications like robotics and autonomous driving, offering an incremental improvement.
The paper tackles plasticity loss in continual learning on edge devices by proposing CBPNet, which improves average accuracy by over 1% on Split CIFAR-100 and achieves state-of-the-art accuracy of 69.41% on Split ImageNet-R with minimal parameter overhead.
To meet the demands of applications like robotics and autonomous driving that require real-time responses to dynamic environments, efficient continual learning methods suitable for edge devices have attracted increasing attention. In this transition, using frozen pretrained models with prompts has become a mainstream strategy to combat catastrophic forgetting. However, this approach introduces a new critical bottleneck: plasticity loss, where the model's ability to learn new knowledge diminishes due to the frozen backbone and the limited capacity of prompt parameters. We argue that the reduction in plasticity stems from a lack of update vitality in underutilized parameters during the training process. To this end, we propose the Continual Backpropagation Prompt Network (CBPNet), an effective and parameter efficient framework designed to restore the model's learning vitality. We innovatively integrate an Efficient CBP Block that counteracts plasticity decay by adaptively reinitializing these underutilized parameters. Experimental results on edge devices demonstrate CBPNet's effectiveness across multiple benchmarks. On Split CIFAR-100, it improves average accuracy by over 1% against a strong baseline, and on the more challenging Split ImageNet-R, it achieves a state of the art accuracy of 69.41%. This is accomplished by training additional parameters that constitute less than 0.2% of the backbone's size, validating our approach.