Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection
This addresses the problem of efficient and private learning on distributed devices with evolving data, representing an incremental advance in federated continual learning.
The paper tackles catastrophic forgetting in federated continual learning by proposing FedProTIP, which projects client updates to reduce interference with prior tasks and includes a task identity prediction mechanism, achieving significant improvements in average accuracy over state-of-the-art methods in task-agnostic settings.
Federated continual learning (FCL) enables distributed client devices to learn from streaming data across diverse and evolving tasks. A major challenge to continual learning, catastrophic forgetting, is exacerbated in decentralized settings by the data heterogeneity, constrained communication and privacy concerns. We propose Federated gradient Projection-based Continual Learning with Task Identity Prediction (FedProTIP), a novel FCL framework that mitigates forgetting by projecting client updates onto the orthogonal complement of the subspace spanned by previously learned representations of the global model. This projection reduces interference with earlier tasks and preserves performance across the task sequence. To further address the challenge of task-agnostic inference, we incorporate a lightweight mechanism that leverages core bases from prior tasks to predict task identity and dynamically adjust the global model's outputs. Extensive experiments across standard FCL benchmarks demonstrate that FedProTIP significantly outperforms state-of-the-art methods in average accuracy, particularly in settings where task identities are a priori unknown.