Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data
This addresses the robustness issue in ABI for probabilistic inverse problems, enabling safer applications in fields like time-series and image analysis, though it is incremental as it builds on existing ABI methods.
The paper tackled the problem of amortized Bayesian inference (ABI) being insufficiently robust for widespread use due to biased posterior approximations on out-of-scope observations, and proposed a semi-supervised approach using unlabeled data with self-consistency losses, resulting in drastically improved robustness and accurate inference even on far-out observations.
Amortized Bayesian inference (ABI) with neural networks can solve probabilistic inverse problems orders of magnitude faster than classical methods. However, ABI is not yet sufficiently robust for widespread and safe application. When performing inference on observations outside the scope of the simulated training data, posterior approximations are likely to become highly biased, which cannot be corrected by additional simulations due to the bad pre-asymptotic behavior of current neural posterior estimators. In this paper, we propose a semi-supervised approach that enables training not only on labeled simulated data generated from the model, but also on \textit{unlabeled} data originating from any source, including real data. To achieve this, we leverage Bayesian self-consistency properties that can be transformed into strictly proper losses that do not require knowledge of ground-truth parameters. We test our approach on several real-world case studies, including applications to high-dimensional time-series and image data. Our results show that semi-supervised learning with unlabeled data drastically improves the robustness of ABI in the out-of-simulation regime. Notably, inference remains accurate even when evaluated on observations far away from the labeled and unlabeled data seen during training.