A variational Bayes approach to debiased inference for low-dimensional parameters in high-dimensional linear regression
This work addresses the challenge of reliable inference in high-dimensional settings for statisticians and data scientists, representing an incremental improvement over existing methods.
The authors tackled the problem of performing accurate statistical inference for low-dimensional parameters in high-dimensional sparse linear regression by proposing a scalable variational Bayes method, which achieves competitive numerical performance and provides theoretical guarantees for estimation and uncertainty quantification.
We propose a scalable variational Bayes method for statistical inference for a single or low-dimensional subset of the coordinates of a high-dimensional parameter in sparse linear regression. Our approach relies on assigning a mean-field approximation to the nuisance coordinates and carefully modelling the conditional distribution of the target given the nuisance. This requires only a preprocessing step and preserves the computational advantages of mean-field variational Bayes, while ensuring accurate and reliable inference for the target parameter, including for uncertainty quantification. We investigate the numerical performance of our algorithm, showing that it performs competitively with existing methods. We further establish accompanying theoretical guarantees for estimation and uncertainty quantification in the form of a Bernstein--von Mises theorem.