KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting
This addresses accuracy issues in federated learning for resource-constrained clients, but it is incremental as it builds on existing SFL frameworks.
The paper tackles the problem of low training accuracy in Split Federated Learning (SFL) due to data heterogeneity and catastrophic forgetting, proposing KoReA-SFL which improves test accuracy by up to 23.25% compared to conventional methods.
Although Split Federated Learning (SFL) is good at enabling knowledge sharing among resource-constrained clients, it suffers from the problem of low training accuracy due to the neglect of data heterogeneity and catastrophic forgetting. To address this issue, we propose a novel SFL approach named KoReA-SFL, which adopts a multi-model aggregation mechanism to alleviate gradient divergence caused by heterogeneous data and a knowledge replay strategy to deal with catastrophic forgetting. Specifically, in KoReA-SFL cloud servers (i.e., fed server and main server) maintain multiple branch model portions rather than a global portion for local training and an aggregated master-model portion for knowledge sharing among branch portions. To avoid catastrophic forgetting, the main server of KoReA-SFL selects multiple assistant devices for knowledge replay according to the training data distribution of each server-side branch-model portion. Experimental results obtained from non-IID and IID scenarios demonstrate that KoReA-SFL significantly outperforms conventional SFL methods (by up to 23.25\% test accuracy improvement).