Exemplar-condensed Federated Class-incremental Learning
This addresses the problem of maintaining model performance across sequential tasks in distributed, privacy-sensitive settings like federated learning, representing an incremental improvement over existing replay-based methods.
The paper tackles catastrophic forgetting in federated continual learning by proposing ECoral, which distills training characteristics from streaming data into informative rehearsal exemplars. The method outperforms several state-of-the-art approaches in experiments.
We propose Exemplar-Condensed federated class-incremental learning (ECoral) to distil the training characteristics of real images from streaming data into informative rehearsal exemplars. The proposed method eliminates the limitations of exemplar selection in replay-based approaches for mitigating catastrophic forgetting in federated continual learning (FCL). The limitations particularly related to the heterogeneity of information density of each summarized data. Our approach maintains the consistency of training gradients and the relationship to past tasks for the summarized exemplars to represent the streaming data compared to the original images effectively. Additionally, our approach reduces the information-level heterogeneity of the summarized data by inter-client sharing of the disentanglement generative model. Extensive experiments show that our ECoral outperforms several state-of-the-art methods and can be seamlessly integrated with many existing approaches to enhance performance.