Federated Coordinate Descent for Privacy-Preserving Multiparty Linear Regression
This addresses privacy-preserving distributed regression for multiparty scenarios, but it is incremental as it extends coordinate descent to a federated context with existing security techniques.
The paper tackles the problem of securely solving L1-regularized linear regression (e.g., Lasso) in multiparty settings, presenting Federated Coordinate Descent (FCD) which achieves privacy via secure aggregation and perturbations, with experimental results showing MAE as low as centralized methods on UCI datasets.
Distributed privacy-preserving regression schemes have been developed and extended in various fields, where multiparty collaboratively and privately run optimization algorithms, e.g., Gradient Descent, to learn a set of optimal parameters. However, traditional Gradient-Descent based methods fail to solve problems which contains objective functions with L1 regularization, such as Lasso regression. In this paper, we present Federated Coordinate Descent, a new distributed scheme called FCD, to address this issue securely under multiparty scenarios. Specifically, through secure aggregation and added perturbations, our scheme guarantees that: (1) no local information is leaked to other parties, and (2) global model parameters are not exposed to cloud servers. The added perturbations can eventually be eliminated by each party to derive a global model with high performance. We show that the FCD scheme fills the gap of multiparty secure Coordinate Descent methods and is applicable for general linear regressions, including linear, ridge and lasso regressions. Theoretical security analysis and experimental results demonstrate that FCD can be performed effectively and efficiently, and provide as low MAE measure as centralized methods under tasks of three types of linear regressions on real-world UCI datasets.