Personalized Federated Learning with Residual Fisher Information for Medical Image Segmentation
This work addresses the problem of data heterogeneity for medical institutions using federated learning, offering an incremental improvement through parameter-level personalization.
The paper tackled data heterogeneity in federated learning for medical image segmentation by proposing pFL-ResFIM, a framework that uses Residual Fisher Information to personalize models at the parameter level, achieving consistent performance improvements over state-of-the-art methods on public datasets.
Federated learning enables multiple clients (institutions) to collaboratively train machine learning models without sharing their private data. To address the challenge of data heterogeneity across clients, personalized federated learning (pFL) aims to learn customized models for each client. In this work, we propose pFL-ResFIM, a novel pFL framework that achieves client-adaptive personalization at the parameter level. Specifically, we introduce a new metric, Residual Fisher Information Matrix (ResFIM), to quantify the sensitivity of model parameters to domain discrepancies. To estimate ResFIM for each client model under privacy constraints, we employ a spectral transfer strategy that generates simulated data reflecting the domain styles of different clients. Based on the estimated ResFIM, we partition model parameters into domain-sensitive and domain-invariant components. A personalized model for each client is then constructed by aggregating only the domain-invariant parameters on the server. Extensive experiments on public datasets demonstrate that pFL-ResFIM consistently outperforms state-of-the-art methods, validating its effectiveness.