Federated Empirical Risk Minimization via Second-Order Method
This work addresses data privacy concerns in distributed machine learning by enabling efficient optimization without sharing raw data, representing an incremental improvement in federated learning methods.
The authors tackled the problem of federated empirical risk minimization by proposing an interior point method, achieving a communication complexity of ̃O(d^{3/2}) per iteration.
Many convex optimization problems with important applications in machine learning are formulated as empirical risk minimization (ERM). There are several examples: linear and logistic regression, LASSO, kernel regression, quantile regression, $p$-norm regression, support vector machines (SVM), and mean-field variational inference. To improve data privacy, federated learning is proposed in machine learning as a framework for training deep learning models on the network edge without sharing data between participating nodes. In this work, we present an interior point method (IPM) to solve a general ERM problem under the federated learning setting. We show that the communication complexity of each iteration of our IPM is $\tilde{O}(d^{3/2})$, where $d$ is the dimension (i.e., number of features) of the dataset.