Continual Horizontal Federated Learning for Heterogeneous Data
This addresses the challenge of data heterogeneity in federated learning for clients with varying feature sets, offering a more efficient collaborative model training approach.
The paper tackles the problem of horizontal federated learning (HFL) under heterogeneous feature spaces, where traditional HFL only uses common features and ignores client-specific ones, by proposing CHFL, a method that splits neural networks into columns for common and unique features, resulting in significant performance improvements over vanilla HFL and local learning on real-world datasets.
Federated learning is a promising machine learning technique that enables multiple clients to collaboratively build a model without revealing the raw data to each other. Among various types of federated learning methods, horizontal federated learning (HFL) is the best-studied category and handles homogeneous feature spaces. However, in the case of heterogeneous feature spaces, HFL uses only common features and leaves client-specific features unutilized. In this paper, we propose a HFL method using neural networks named continual horizontal federated learning (CHFL), a continual learning approach to improve the performance of HFL by taking advantage of unique features of each client. CHFL splits the network into two columns corresponding to common features and unique features, respectively. It jointly trains the first column by using common features through vanilla HFL and locally trains the second column by using unique features and leveraging the knowledge of the first one via lateral connections without interfering with the federated training of it. We conduct experiments on various real world datasets and show that CHFL greatly outperforms vanilla HFL that only uses common features and local learning that uses all features that each client has.