Balancing Privacy and Performance for Private Federated Learning Algorithms
This work addresses the challenge of maintaining data privacy without sacrificing model performance for users of federated learning systems, representing an incremental improvement by optimizing existing methods.
The paper tackles the trade-off between privacy and performance in federated learning by determining the optimal balance between local steps and communication rounds to maximize convergence under a differential privacy budget, proving and empirically validating that this optimal configuration depends on privacy parameters and problem variables for strongly convex optimization.
Federated learning (FL) is a distributed machine learning (ML) framework where multiple clients collaborate to train a model without exposing their private data. FL involves cycles of local computations and bi-directional communications between the clients and server. To bolster data security during this process, FL algorithms frequently employ a differential privacy (DP) mechanism that introduces noise into each client's model updates before sharing. However, while enhancing privacy, the DP mechanism often hampers convergence performance. In this paper, we posit that an optimal balance exists between the number of local steps and communication rounds, one that maximizes the convergence performance within a given privacy budget. Specifically, we present a proof for the optimal number of local steps and communication rounds that enhance the convergence bounds of the DP version of the ScaffNew algorithm. Our findings reveal a direct correlation between the optimal number of local steps, communication rounds, and a set of variables, e.g the DP privacy budget and other problem parameters, specifically in the context of strongly convex optimization. We furthermore provide empirical evidence to validate our theoretical findings.