Prototype Augmented Hypernetworks for Continual Learning
This addresses the problem of forgetting prior knowledge in sequential task learning for AI systems, representing a strong incremental improvement over existing methods.
The paper tackles catastrophic forgetting in continual learning by introducing Prototype-Augmented Hypernetworks (PAH), which uses a hypernetwork with task prototypes to generate task-specific classifiers and achieves state-of-the-art results, such as 74.5% accuracy with 1.7% forgetting on Split-CIFAR100.
Continual learning (CL) aims to learn a sequence of tasks without forgetting prior knowledge, but gradient updates for a new task often overwrite the weights learned earlier, causing catastrophic forgetting (CF). We propose Prototype-Augmented Hypernetworks (PAH), a framework where a single hypernetwork, conditioned on learnable task prototypes, dynamically generates task-specific classifier heads on demand. To mitigate forgetting, PAH combines cross-entropy with dual distillation losses, one to align logits and another to align prototypes, ensuring stable feature representations across tasks. Evaluations on Split-CIFAR100 and TinyImageNet demonstrate that PAH achieves state-of-the-art performance, reaching 74.5 % and 63.7 % accuracy with only 1.7 % and 4.4 % forgetting, respectively, surpassing prior methods without storing samples or heads.