FedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling
For federated learning with heterogeneous clients and resource-constrained devices, FedOBP provides a theoretically grounded method to efficiently select which parameters to personalize, reducing computational burden on clients.
FedOBP addresses the challenge of balancing global knowledge sharing and local adaptation in personalized federated learning by introducing an element-wise importance score based on Optimal Brain Damage pruning theory. It achieves state-of-the-art performance across diverse datasets and heterogeneity scenarios while personalizing only a very small number of parameters.
Federated Learning (FL) faces challenges from client data heterogeneity and resource-constrained mobile devices, which can degrade model accuracy. Personalized Federated Learning (PFL) addresses this issue by adapting shared global knowledge to local data distributions. A promising approach in PFL is model decoupling, which separates the model into global and personalized parameters, raising the key question of which parameters should be personalized to balance global knowledge sharing and local adaptation. In this paper, we propose a Federated Optimal Brain Personalization (FedOBP) algorithm with a quantile-based thresholding mechanism and introduce an element-wise importance score. This score extends Optimal Brain Damage (OBD) pruning theory by incorporating a federated approximation of the first-order derivative in the Taylor expansion to evaluate the importance of each parameter for personalization. Moreover, we move the metric computation originally performed on clients to the server side, to alleviate the burden on resource-constrained mobile devices. To the best of our knowledge, this is the first work to bridge classical saliency-based pruning theory with federated parameter decoupling, providing a rigorous theoretical justification for selecting personalized parameters based on their sensitivity to local loss landscapes. Extensive experiments demonstrate that FedOBP outperforms state-of-the-art methods across diverse datasets and heterogeneity scenarios, while requiring personalization of only a very small number of personalized parameters.