Communication-efficient Distributed Cooperative Learning with Compressed Beliefs
This addresses communication efficiency in distributed learning for multi-agent systems, offering incremental improvements over existing methods.
The paper tackles the problem of distributed cooperative learning among agents with large hypothesis sets by proposing a belief update rule that uses compressed (sparse or quantized) beliefs to reduce communication. The result shows almost sure asymptotic exponential convergence and a linear concentration rate, with simulations reducing transmitted bits to 5-10% of non-compressed methods.
We study the problem of distributed cooperative learning, where a group of agents seeks to agree on a set of hypotheses that best describes a sequence of private observations. In the scenario where the set of hypotheses is large, we propose a belief update rule where agents share compressed (either sparse or quantized) beliefs with an arbitrary positive compression rate. Our algorithm leverages a unified communication rule that enables agents to access wide-ranging compression operators as black-box modules. We prove the almost sure asymptotic exponential convergence of beliefs around the set of optimal hypotheses. Additionally, we show a non-asymptotic, explicit, and linear concentration rate in probability of the beliefs on the optimal hypothesis set. We provide numerical experiments to illustrate the communication benefits of our method. The simulation results show that the number of transmitted bits can be reduced to 5-10% of the non-compressed method in the studied scenarios.