Sparse Horseshoe Estimation via Expectation-Maximisation
This work addresses the challenge of sparse estimation in Bayesian statistics, offering a practical solution for researchers and practitioners in fields requiring interpretable models, though it is incremental as it builds on existing horseshoe prior methods.
The authors tackled the problem of obtaining sparse parameter estimates using the horseshoe prior in Bayesian linear models, where conventional posterior mean estimates are not sparse. They proposed a novel expectation-maximisation (EM) procedure for computing maximum a posteriori (MAP) estimates, achieving performance comparable or superior to state-of-the-art methods in statistical performance and computational cost on simulated and real data.
The horseshoe prior is known to possess many desirable properties for Bayesian estimation of sparse parameter vectors, yet its density function lacks an analytic form. As such, it is challenging to find a closed-form solution for the posterior mode. Conventional horseshoe estimators use the posterior mean to estimate the parameters, but these estimates are not sparse. We propose a novel expectation-maximisation (EM) procedure for computing the MAP estimates of the parameters in the case of the standard linear model. A particular strength of our approach is that the M-step depends only on the form of the prior and it is independent of the form of the likelihood. We introduce several simple modifications of this EM procedure that allow for straightforward extension to generalised linear models. In experiments performed on simulated and real data, our approach performs comparable, or superior to, state-of-the-art sparse estimation methods in terms of statistical performance and computational cost.