Conformalized Neural Networks for Federated Uncertainty Quantification under Dual Heterogeneity
This work provides a method for reliable uncertainty quantification in federated learning, which is crucial for preventing overconfident model deployments and silent local failures for under-resourced agents.
This paper addresses the challenge of uncertainty quantification (UQ) in federated learning (FL) under both data and model heterogeneity. The proposed method, FedWQ-CP, achieves reliable agent-wise and global coverage while generating the smallest prediction sets or intervals across seven public datasets for classification and regression.
Federated learning (FL) faces challenges in uncertainty quantification (UQ). Without reliable UQ, FL systems risk deploying overconfident models at under-resourced agents, leading to silent local failures despite seemingly satisfactory global performance. Existing federated UQ approaches often address data heterogeneity or model heterogeneity in isolation, overlooking their joint effect on coverage reliability across agents. Conformal prediction is a widely used distribution-free UQ framework, yet its applications in heterogeneous FL settings remains underexplored. We provide FedWQ-CP, a simple yet effective approach that balances empirical coverage performance with efficiency at both global and agent levels under the dual heterogeneity. FedWQ-CP performs agent-server calibration in a single communication round. On each agent, conformity scores are computed on calibration data and a local quantile threshold is derived. Each agent then transmits only its quantile threshold and calibration sample size to the server. The server simply aggregates these thresholds through a weighted average to produce a global threshold. Experimental results on seven public datasets for both classification and regression demonstrate that FedWQ-CP empirically maintains agent-wise and global coverage while producing the smallest prediction sets or intervals.