ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression
This work addresses the problem of scalable and communication-efficient sampling in federated learning for the ML/statistics community, presenting incremental improvements over existing federated Langevin methods.
The paper tackles the challenge of efficient federated sampling by proposing three compressed Langevin algorithms (P-ELF, D-ELF, B-ELF) with primal, dual, and bidirectional compression, and provides non-asymptotic convergence guarantees under Log-Sobolev inequality.
Federated sampling algorithms have recently gained great popularity in the community of machine learning and statistics. This paper studies variants of such algorithms called Error Feedback Langevin algorithms (ELF). In particular, we analyze the combinations of EF21 and EF21-P with the federated Langevin Monte-Carlo. We propose three algorithms: P-ELF, D-ELF, and B-ELF that use, respectively, primal, dual, and bidirectional compressors. We analyze the proposed methods under Log-Sobolev inequality and provide non-asymptotic convergence guarantees.