Efficient Distributed Estimation of Inverse Covariance Matrices
This work addresses communication efficiency for distributed statistical estimation, which is an incremental improvement in a domain-specific context.
The paper tackles the problem of estimating sparse inverse covariance matrices in distributed systems by proposing a communication-efficient method that transfers only a small subset of entries per machine in a single round, achieving error rates comparable to non-distributed settings and enabling correct model selection, as demonstrated through simulations.
In distributed systems, communication is a major concern due to issues such as its vulnerability or efficiency. In this paper, we are interested in estimating sparse inverse covariance matrices when samples are distributed into different machines. We address communication efficiency by proposing a method where, in a single round of communication, each machine transfers a small subset of the entries of the inverse covariance matrix. We show that, with this efficient distributed method, the error rates can be comparable with estimation in a non-distributed setting, and correct model selection is still possible. Practical performance is shown through simulations.