pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup
This addresses personalized federated learning for non-IID data scenarios, offering an incremental improvement over existing methods.
The paper tackled the problem of model accuracy degradation in personalized federated learning due to global-local model discrepancy, client drift, and catastrophic forgetting, and proposed pMixFed, which outperformed state-of-the-art methods with faster training, increased robustness, and improved handling of data heterogeneity.
Traditional Federated Learning (FL) methods encounter significant challenges when dealing with heterogeneous data and providing personalized solutions for non-IID scenarios. Personalized Federated Learning (PFL) approaches aim to address these issues by balancing generalization and personalization, often through parameter decoupling or partial models that freeze some neural network layers for personalization while aggregating other layers globally. However, existing methods still face challenges of global-local model discrepancy, client drift, and catastrophic forgetting, which degrade model accuracy. To overcome these limitations, we propose $\textit{pMixFed}$, a dynamic, layer-wise PFL approach that integrates $\textit{mixup}$ between shared global and personalized local models. Our method introduces an adaptive strategy for partitioning between personalized and shared layers, a gradual transition of personalization degree to enhance local client adaptation, improved generalization across clients, and a novel aggregation mechanism to mitigate catastrophic forgetting. Extensive experiments demonstrate that pMixFed outperforms state-of-the-art PFL methods, showing faster model training, increased robustness, and improved handling of data heterogeneity under different heterogeneous settings.