Distribution-Free Federated Learning with Conformal Predictions
This addresses challenges in deploying federated models into clinical practice for healthcare applications, but it is incremental as it adapts existing conformal methods to the federated setting.
The paper tackled the problem of poor calibration and lack of interpretability in federated learning for healthcare by incorporating an adaptive conformal framework to ensure distribution-free prediction sets with coverage guarantees, resulting in tighter coverage over local conformal predictions on 6 medical imaging datasets for 2D and 3D multi-class classification tasks.
Federated learning has attracted considerable interest for collaborative machine learning in healthcare to leverage separate institutional datasets while maintaining patient privacy. However, additional challenges such as poor calibration and lack of interpretability may also hamper widespread deployment of federated models into clinical practice, leading to user distrust or misuse of ML tools in high-stakes clinical decision-making. In this paper, we propose to address these challenges by incorporating an adaptive conformal framework into federated learning to ensure distribution-free prediction sets that provide coverage guarantees. Importantly, these uncertainty estimates can be obtained without requiring any additional modifications to the model. Empirical results on the MedMNIST medical imaging benchmark demonstrate our federated method provides tighter coverage over local conformal predictions on 6 different medical imaging datasets for 2D and 3D multi-class classification tasks. Furthermore, we correlate class entropy with prediction set size to assess task uncertainty.