Unsupervised Continual Learning for Amortized Bayesian Inference
This work provides a viable path for trustworthy Amortized Bayesian Inference for practitioners dealing with real-world, sequentially arriving data and distribution shifts, improving the robustness and accuracy of posterior estimates.
This paper addresses the problem of performance degradation in Amortized Bayesian Inference (ABI) when faced with model misspecification and sequentially arriving data. The authors propose a continual learning framework that significantly mitigates catastrophic forgetting and yields posterior estimates closer to MCMC references across three diverse case studies, outperforming standard simulation-based training.
Amortized Bayesian Inference (ABI) enables efficient posterior estimation using generative neural networks trained on simulated data, but often suffers from performance degradation under model misspecification. While self-consistency (SC) training on unlabeled empirical data can enhance network robustness, current approaches are limited to static, single-task settings and fail to handle sequentially arriving data or distribution shifts. We propose a continual learning framework for ABI that decouples simulation-based pre-training from unsupervised sequential SC fine-tuning on real-world data. To address the challenge of catastrophic forgetting, we introduce two adaptation strategies: (1) SC with episodic replay, utilizing a memory buffer of past observations, and (2) SC with elastic weight consolidation, which regularizes updates to preserve task-critical parameters. Across three diverse case studies, our methods significantly mitigate forgetting and yield posterior estimates that outperform standard simulation-based training, achieving estimates closer to MCMC reference, providing a viable path for trustworthy ABI across a range of different tasks.