Communication-Efficient Distributed Statistical Inference
This work addresses communication bottlenecks in distributed statistical inference, offering incremental improvements across multiple inference tasks.
The paper tackles distributed statistical inference by proposing a Communication-efficient Surrogate Likelihood (CSL) framework, which improves upon naive averaging for low-dimensional estimation, achieves minimax-optimal results for high-dimensional regularized estimation, and enhances computational efficiency for Bayesian inference, including in non-distributed settings.
We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For low-dimensional estimation, CSL provably improves upon naive averaging schemes and facilitates the construction of confidence intervals. For high-dimensional regularized estimation, CSL leads to a minimax-optimal estimator with controlled communication cost. For Bayesian inference, CSL can be used to form a communication-efficient quasi-posterior distribution that converges to the true posterior. This quasi-posterior procedure significantly improves the computational efficiency of MCMC algorithms even in a non-distributed setting. We present both theoretical analysis and experiments to explore the properties of the CSL approximation.