Privacy-Aware Compression for Federated Learning Through Numerical Mechanism Design
This work addresses privacy and communication bottlenecks in federated learning for distributed systems, representing an incremental improvement over prior methods.
The paper tackles the challenge of balancing privacy, accuracy, and communication efficiency in federated learning by introducing an Interpolated MVU mechanism, which improves scalability and achieves state-of-the-art results on various datasets.
In private federated learning (FL), a server aggregates differentially private updates from a large number of clients in order to train a machine learning model. The main challenge in this setting is balancing privacy with both classification accuracy of the learnt model as well as the number of bits communicated between the clients and server. Prior work has achieved a good trade-off by designing a privacy-aware compression mechanism, called the minimum variance unbiased (MVU) mechanism, that numerically solves an optimization problem to determine the parameters of the mechanism. This paper builds upon it by introducing a new interpolation procedure in the numerical design process that allows for a far more efficient privacy analysis. The result is the new Interpolated MVU mechanism that is more scalable, has a better privacy-utility trade-off, and provides SOTA results on communication-efficient private FL on a variety of datasets.