Pre-trained Tabular Foundation Models as Versatile Summary Networks for Neural Posterior Estimation
This work addresses the need for training-free summary networks in simulation-based inference, offering a modular approach that leverages pretrained tabular foundation models.
The authors propose PFN-NPE, a method that uses a pretrained TabPFN as a fixed summary network for simulation-based Bayesian inference, paired with a downstream inference head. PFN-NPE matches or outperforms established posterior approximation methods across diverse settings, though it struggles with joint posterior structure.
In this work, we study TabPFN as a training-free, modular summary network for simulation-based Bayesian inference (SBI). Tabular foundation models such as TabPFN are pretrained on broad families of synthetic tabular data-generating processes and adapt at test time through in-context learning, making them natural candidates for SBI, where posterior estimation often depends on learning informative summaries of simulated observations. We propose PFN-NPE: a general recipe that uses a pretrained TabPFN encoder as a fixed summary network for simulator outputs, then pairs the resulting summaries with a downstream inference head chosen for the problem. With normalizing flows as the default inference head, PFN-NPE matches established posterior approximation methods and sometimes outperforms them. More importantly, diagnostic probes show that the TabPFN-derived summaries often preserve useful posterior location and marginal information. These analyses also reveal a limitation in that TabPFN-derived summaries may struggle to represent the joint posterior structure even when the marginals are well recovered. Still, our experiments show that TabPFN can serve as an effective summary network across a diverse set of SBI settings, with the inference network left modular and task-dependent.