Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype
This addresses scalability issues in continual learning for AI systems, though it appears incremental as it builds on existing prompt-based methods.
The paper tackles the problem of inter-task interference and scalability in continual learning by proposing a key-value pair-free method using task-specific Prompt-Prototype (ProP), which eliminates key-value pairs and shows effectiveness in experiments on widely used datasets.
Continual learning aims to enable models to acquire new knowledge while retaining previously learned information. Prompt-based methods have shown remarkable performance in this domain; however, they typically rely on key-value pairing, which can introduce inter-task interference and hinder scalability. To overcome these limitations, we propose a novel approach employing task-specific Prompt-Prototype (ProP), thereby eliminating the need for key-value pairs. In our method, task-specific prompts facilitate more effective feature learning for the current task, while corresponding prototypes capture the representative features of the input. During inference, predictions are generated by binding each task-specific prompt with its associated prototype. Additionally, we introduce regularization constraints during prompt initialization to penalize excessively large values, thereby enhancing stability. Experiments on several widely used datasets demonstrate the effectiveness of the proposed method. In contrast to mainstream prompt-based approaches, our framework removes the dependency on key-value pairs, offering a fresh perspective for future continual learning research.