Variational approximations of empirical Bayes posteriors in high-dimensional linear models
This work addresses computational bottlenecks for researchers and practitioners using high-dimensional linear models, offering a faster alternative to MCMC while maintaining statistical guarantees, though it is incremental as it builds on prior empirical Bayes approaches.
The paper tackles the computational expense of Markov chain Monte Carlo methods for empirical Bayes posteriors in high-dimensional linear models by developing a variational approximation that is fast to compute and retains optimal concentration rates. In simulations, the method demonstrates superior performance compared to existing variational approximations across various high-dimensional settings.
In high-dimensions, the prior tails can have a significant effect on both posterior computation and asymptotic concentration rates. To achieve optimal rates while keeping the posterior computations relatively simple, an empirical Bayes approach has recently been proposed, featuring thin-tailed conjugate priors with data-driven centers. While conjugate priors ease some of the computational burden, Markov chain Monte Carlo methods are still needed, which can be expensive when dimension is high. In this paper, we develop a variational approximation to the empirical Bayes posterior that is fast to compute and retains the optimal concentration rate properties of the original. In simulations, our method is shown to have superior performance compared to existing variational approximations in the literature across a wide range of high-dimensional settings.