Asynchronous Federated Continual Learning
This addresses the challenge of realistic continual learning in federated environments for distributed systems, though it is incremental in adapting existing methods to a new setting.
The paper tackles the problem of asynchronous federated continual learning, where clients learn tasks independently with different orders and timings, by introducing a novel setting (AFCL) and proposing FedSpace with prototype-based learning and other techniques, achieving effective results on CIFAR-100 with up to 500 clients.
The standard class-incremental continual learning setting assumes a set of tasks seen one after the other in a fixed and predefined order. This is not very realistic in federated learning environments where each client works independently in an asynchronous manner getting data for the different tasks in time-frames and orders totally uncorrelated with the other ones. We introduce a novel federated learning setting (AFCL) where the continual learning of multiple tasks happens at each client with different orderings and in asynchronous time slots. We tackle this novel task using prototype-based learning, a representation loss, fractal pre-training, and a modified aggregation policy. Our approach, called FedSpace, effectively tackles this task as shown by the results on the CIFAR-100 dataset using 3 different federated splits with 50, 100, and 500 clients, respectively. The code and federated splits are available at https://github.com/LTTM/FedSpace.