KAC: Kolmogorov-Arnold Classifier for Continual Learning
This addresses the issue of forgetting in continual learning for AI systems, but it is incremental as it builds on existing methods by integrating KAN structures.
The paper tackles the problem of continual learning by introducing the Kolmogorov-Arnold Classifier (KAC), which replaces linear classifiers to improve stability and performance across tasks, demonstrating performance improvements in various benchmarks.
Continual learning requires models to train continuously across consecutive tasks without forgetting. Most existing methods utilize linear classifiers, which struggle to maintain a stable classification space while learning new tasks. Inspired by the success of Kolmogorov-Arnold Networks (KAN) in preserving learning stability during simple continual regression tasks, we set out to explore their potential in more complex continual learning scenarios. In this paper, we introduce the Kolmogorov-Arnold Classifier (KAC), a novel classifier developed for continual learning based on the KAN structure. We delve into the impact of KAN's spline functions and introduce Radial Basis Functions (RBF) for improved compatibility with continual learning. We replace linear classifiers with KAC in several recent approaches and conduct experiments across various continual learning benchmarks, all of which demonstrate performance improvements, highlighting the effectiveness and robustness of KAC in continual learning. The code is available at https://github.com/Ethanhuhuhu/KAC.