APFL: Analytic Personalized Federated Learning via Dual-Stream Least Squares
This addresses the problem of data heterogeneity for federated learning practitioners, offering a novel method with theoretical guarantees, though it is incremental in building on existing PFL frameworks.
The paper tackles the challenge of non-IID data in Personalized Federated Learning (PFL) by proposing APFL, an analytic approach using dual-stream least squares, which achieves accuracy improvements of 1.10%-15.45% over state-of-the-art baselines.
Personalized Federated Learning (PFL) has presented a significant challenge to deliver personalized models to individual clients through collaborative training. Existing PFL methods are often vulnerable to non-IID data, which severely hinders collective generalization and then compromises the subsequent personalization efforts. In this paper, to address this non-IID issue in PFL, we propose an Analytic Personalized Federated Learning (APFL) approach via dual-stream least squares. In our APFL, we use a foundation model as a frozen backbone for feature extraction. Subsequent to the feature extractor, we develop dual-stream analytic models to achieve both collective generalization and individual personalization. Specifically, our APFL incorporates a shared primary stream for global generalization across all clients, and a dedicated refinement stream for local personalization of each individual client. The analytical solutions of our APFL enable its ideal property of heterogeneity invariance, theoretically meaning that each personalized model remains identical regardless of how heterogeneous the data are distributed across all other clients. Empirical results across various datasets also validate the superiority of our APFL over state-of-the-art baselines, with advantages of at least 1.10%-15.45% in accuracy.