On the Uncertainty Quantification Ability of Tabular Foundation Models
For practitioners in mechanics and computational science needing reliable UQ, this study reveals a trade-off between explicit and learned priors, guiding model selection based on data availability and problem complexity.
This paper compares the uncertainty quantification (UQ) capabilities of tabular foundation model TabPFN against Gaussian processes (GPs) across regression tasks. TabPFN excels in complex, high-dimensional problems with sufficient data, while GPs provide better predictive accuracy and UQ in data-scarce settings or when the kernel matches the prior.
Foundation models (FMs) have achieved substantial success in generalizing across tasks without problemspecific training or fine-tuning. However, many critical applications in mechanics and computational science require not only accurate predictions but also reliable uncertainty quantification (UQ). Herein we investigate the UQ capabilities of tabular FMs in regression tasks through a comprehensive empirical study comparing Tabular Prior-Data Fitted Networks (TabPFN) against Gaussian processes (GPs). We systematically evaluate these two methods across a host of regression problems with varying complexity, dataset sizes, and input dimensionalities. We use a default setting to build all the GPs and for a fair comparison against TabPFN v2.5. Our findings highlight an important trade-off between explicit and learned priors: while TabPFN achieves highly competitive performance for complex, high-dimensional problems with sufficient data, GPs often provide superior predictive accuracy and UQ in data-scarce settings. Moreover, when the chosen kernel constitutes a good prior for the underlying function, GP performance can substantially exceed that of TabPFN. Our results can be reproduced from https://github.com/kianswarehouse/GPvsPFN.