A Primal-Dual Algorithm for Hybrid Federated Learning
This addresses a practical scenario in federated learning with incremental improvements for applications requiring data privacy and distributed feature sets.
The paper tackles the problem of hybrid federated learning, where clients hold subsets of both features and samples, by proposing a fast, robust algorithm based on Fenchel Duality, which converges to a centralized solution and outperforms FedAvg and HyFEM in experiments.
Very few methods for hybrid federated learning, where clients only hold subsets of both features and samples, exist. Yet, this scenario is extremely important in practical settings. We provide a fast, robust algorithm for hybrid federated learning that hinges on Fenchel Duality. We prove the convergence of the algorithm to the same solution as if the model is trained centrally in a variety of practical regimes. Furthermore, we provide experimental results that demonstrate the performance improvements of the algorithm over a commonly used method in federated learning, FedAvg, and an existing hybrid FL algorithm, HyFEM. We also provide privacy considerations and necessary steps to protect client data.