Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework
This work addresses robustness issues in federated learning for applications where data is distributed, offering a solution to prior and likelihood misspecification.
The paper tackles the problem of model misspecification in federated learning by introducing FedGVI, a probabilistic framework that provides unbiased predictions with calibrated uncertainty, showing improved robustness and predictive performance on synthetic and real-world datasets.
We introduce FedGVI, a probabilistic Federated Learning (FL) framework that is robust to both prior and likelihood misspecification. FedGVI addresses limitations in both frequentist and Bayesian FL by providing unbiased predictions under model misspecification, with calibrated uncertainty quantification. Our approach generalises previous FL approaches, specifically Partitioned Variational Inference (Ashman et al., 2022), by allowing robust and conjugate updates, decreasing computational complexity at the clients. We offer theoretical analysis in terms of fixed-point convergence, optimality of the cavity distribution, and provable robustness to likelihood misspecification. Further, we empirically demonstrate the effectiveness of FedGVI in terms of improved robustness and predictive performance on multiple synthetic and real world classification data sets.