Nearly Optimal Variational Inference for High Dimensional Regression with Shrinkage Priors
This provides a computationally efficient alternative to MCMC for high-dimensional regression, benefiting applications like massive data analysis, though it is incremental as it builds on existing VB and shrinkage prior frameworks.
The authors tackled the problem of high-dimensional linear model inference with heavy tail shrinkage priors by proposing a variational Bayesian (VB) procedure, achieving nearly optimal contraction rates, shorter computing time, higher estimation accuracy, and lower variable selection error compared to competitive methods.
We propose a variational Bayesian (VB) procedure for high-dimensional linear model inferences with heavy tail shrinkage priors, such as student-t prior. Theoretically, we establish the consistency of the proposed VB method and prove that under the proper choice of prior specifications, the contraction rate of the VB posterior is nearly optimal. It justifies the validity of VB inference as an alternative of Markov Chain Monte Carlo (MCMC) sampling. Meanwhile, comparing to conventional MCMC methods, the VB procedure achieves much higher computational efficiency, which greatly alleviates the computing burden for modern machine learning applications such as massive data analysis. Through numerical studies, we demonstrate that the proposed VB method leads to shorter computing time, higher estimation accuracy, and lower variable selection error than competitive sparse Bayesian methods.