Bayesian Semi-structured Subspace Inference
This work addresses the lack of epistemic uncertainty quantification in semi-structured models, which combine interpretable structured and flexible unstructured components, offering a solution for domains requiring both interpretability and uncertainty estimates.
The paper tackled the problem of accounting for epistemic uncertainty in semi-structured regression models by introducing a Bayesian approximation using subspace inference, achieving competitive predictive performance and recovering structured effect parameter posteriors that approach full-space MCMC results as subspace dimension increases.
Semi-structured regression models enable the joint modeling of interpretable structured and complex unstructured feature effects. The structured model part is inspired by statistical models and can be used to infer the input-output relationship for features of particular importance. The complex unstructured part defines an arbitrary deep neural network and thereby provides enough flexibility to achieve competitive prediction performance. While these models can also account for aleatoric uncertainty, there is still a lack of work on accounting for epistemic uncertainty. In this paper, we address this problem by presenting a Bayesian approximation for semi-structured regression models using subspace inference. To this end, we extend subspace inference for joint posterior sampling from a full parameter space for structured effects and a subspace for unstructured effects. Apart from this hybrid sampling scheme, our method allows for tunable complexity of the subspace and can capture multiple minima in the loss landscape. Numerical experiments validate our approach's efficacy in recovering structured effect parameter posteriors in semi-structured models and approaching the full-space posterior distribution of MCMC for increasing subspace dimension. Further, our approach exhibits competitive predictive performance across simulated and real-world datasets.