Analytic solution and stationary phase approximation for the Bayesian lasso and elastic net
This work addresses a bottleneck in Bayesian inference for statisticians and data scientists by enabling more efficient and scalable variable selection in high-dimensional linear regression, though it is incremental as it builds on existing methods with a novel analytical approach.
The paper tackled the problem of analytically intractable partition function integrals in Bayesian lasso and elastic net models by using Fourier transforms to express them as oscillatory integrals, resulting in an analytic expansion and stationary phase approximation that provides highly accurate numerical estimates for posterior distributions and enables Bayesian inference in higher-dimensional models.
The lasso and elastic net linear regression models impose a double-exponential prior distribution on the model parameters to achieve regression shrinkage and variable selection, allowing the inference of robust models from large data sets. However, there has been limited success in deriving estimates for the full posterior distribution of regression coefficients in these models, due to a need to evaluate analytically intractable partition function integrals. Here, the Fourier transform is used to express these integrals as complex-valued oscillatory integrals over "regression frequencies". This results in an analytic expansion and stationary phase approximation for the partition functions of the Bayesian lasso and elastic net, where the non-differentiability of the double-exponential prior has so far eluded such an approach. Use of this approximation leads to highly accurate numerical estimates for the expectation values and marginal posterior distributions of the regression coefficients, and allows for Bayesian inference of much higher dimensional models than previously possible.