Federated Continual Learning: Concepts, Challenges, and Solutions
It tackles the problem of enabling efficient and private model training in dynamic, distributed environments for applications like IoT and mobile computing, but is incremental as it synthesizes existing concepts.
This survey reviews Federated Continual Learning (FCL), addressing challenges like heterogeneity and catastrophic forgetting in collaborative training across distributed devices, and integrates insights from federated and continual learning to improve system efficacy and scalability.
Federated Continual Learning (FCL) has emerged as a robust solution for collaborative model training in dynamic environments, where data samples are continuously generated and distributed across multiple devices. This survey provides a comprehensive review of FCL, focusing on key challenges such as heterogeneity, model stability, communication overhead, and privacy preservation. We explore various forms of heterogeneity and their impact on model performance. Solutions to non-IID data, resource-constrained platforms, and personalized learning are reviewed in an effort to show the complexities of handling heterogeneous data distributions. Next, we review techniques for ensuring model stability and avoiding catastrophic forgetting, which are critical in non-stationary environments. Privacy-preserving techniques are another aspect of FCL that have been reviewed in this work. This survey has integrated insights from federated learning and continual learning to present strategies for improving the efficacy and scalability of FCL systems, making it applicable to a wide range of real-world scenarios.