FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentation
This work provides a more robust and reliable federated fine-tuning approach for remote sensing image segmentation, which is beneficial for applications requiring collaborative model training on distributed and private datasets, representing an incremental improvement in federated learning for computer vision.
This paper addresses the challenge of unreliable collaborative optimization in federated remote sensing image segmentation (RSIS) due to update uncertainty from heterogeneous client data. The authors propose FedEU, a framework that uses personalized evidential uncertainty modeling to quantify epistemic variations and identify high-risk areas, along with client-specific feature embedding to enhance representation. This approach enables adaptive global aggregation via a Top-k uncertainty-guided weighting strategy, leading to superior performance across three large-scale heterogeneous datasets.
Remote sensing image segmentation (RSIS) in federated environments has gained increasing attention because it enables collaborative model training across distributed datasets without sharing raw imagery or annotations. Federated RSIS combined with parameter-efficient fine-tuning (PEFT) can unleash the generalization power of pretrained foundation models for real-world applications, with minimal parameter aggregation and communication overhead. However, the dynamic adaptation of pretrained models to heterogeneous client data inevitably increases update uncertainty and compromises the reliability of collaborative optimization due to the lack of uncertainty estimation for each local model. To bridge this gap, we present FedEU, a federated optimization framework for fine-tuning RSIS models driven by evidential uncertainty. Specifically, personalized evidential uncertainty modeling is introduced to quantify epistemic variations of local models and identify high-risk areas under local data distributions. Furthermore, the client-specific feature embedding (CFE) is exploited to enhance channel-aware feature representation while preserving client-specific properties through personalized attention and an element-aware parameter update approach. These uncertainty estimates are uploaded to the server to enable adaptive global aggregation via a Top-k uncertainty-guided weighting (TUW) strategy, which mitigates the impact of distribution shifts and unreliable updates. Extensive experiments on three large-scale heterogeneous datasets demonstrate the superior performance of FedEU. More importantly, FedEU enables balanced model adaptation across diverse clients by explicitly reducing prediction uncertainty, resulting in more robust and reliable federated outcomes. The source codes will be available at https://github.com/zxk688/FedEU.