Federated Martingale Posterior Samping
For federated learning practitioners, FMP provides a practical Bayesian approach that eliminates prior misspecification while maintaining privacy and communication efficiency.
The paper proposes federated martingale posterior (FMP) sampling, a one-shot embarrassingly parallel protocol for Bayesian neural networks that avoids prior specification by using predictive distributions. Experiments on MNIST, CIFAR-10, and CIFAR-100 show FMP closely matches centralized performance and improves calibration over consensus baselines.
Federated Bayesian neural networks require fixing a prior on the model parameters together with a likelihood. Eliciting meaningful priors on the weight space of modern overparameterized models is notoriously difficult, and misspecification of either component can severely degrade accuracy and calibration. Motivated by the rapid progress of predictive models such as large language models, the martingale posterior, also known as predictive Bayes, replaces the prior--likelihood pair with a predictive distribution and recovers parameter uncertainty by repeatedly drawing predictive samples and refitting the model. A direct federated implementation, however, would require clients to share the local data sets. This letter proposes {federated martingale posterior} (FMP) sampling, a one-shot embarrassingly parallel protocol in which each client uploads a small set of trainable data embeddings and the server runs the predictive sampler centrally. Experiments on MNIST, CIFAR-10, and CIFAR-100 show that FMP closely matches the centralized counterpart and significantly improves calibration over consensus-style baselines.