Hybrid Federated Learning: Algorithms and Implementation
This work tackles the practical problem of hybrid federated learning for organizations with partially overlapping datasets, providing a foundational formulation and algorithm where none existed before.
This paper addresses the less-explored hybrid federated learning (FL) setting, which involves partially overlapped feature and sample spaces. The authors propose a new model-matching-based problem formulation and an efficient algorithm to collaboratively train global and local models for full and partially featured data.
Federated learning (FL) is a recently proposed distributed machine learning paradigm dealing with distributed and private data sets. Based on the data partition pattern, FL is often categorized into horizontal, vertical, and hybrid settings. Despite the fact that many works have been developed for the first two approaches, the hybrid FL setting (which deals with partially overlapped feature space and sample space) remains less explored, though this setting is extremely important in practice. In this paper, we first set up a new model-matching-based problem formulation for hybrid FL, then propose an efficient algorithm that can collaboratively train the global and local models to deal with full and partial featured data. We conduct numerical experiments on the multi-view ModelNet40 data set to validate the performance of the proposed algorithm. To the best of our knowledge, this is the first formulation and algorithm developed for the hybrid FL.