Completely Heterogeneous Federated Learning
This addresses privacy and heterogeneity challenges in federated learning for applications requiring strict data confidentiality, though it appears incremental as it builds on existing FL methods.
The paper tackles the problem of federated learning under three constraints—cross-domain, heterogeneous models, and non-i.i.d. labels—by proposing a 'completely heterogeneous' scenario where clients expose no private information, and it achieves high performance where other methods fail.
Federated learning (FL) faces three major difficulties: cross-domain, heterogeneous models, and non-i.i.d. labels scenarios. Existing FL methods fail to handle the above three constraints at the same time, and the level of privacy protection needs to be lowered (e.g., the model architecture and data category distribution can be shared). In this work, we propose the challenging "completely heterogeneous" scenario in FL, which refers to that each client will not expose any private information including feature space, model architecture, and label distribution. We then devise an FL framework based on parameter decoupling and data-free knowledge distillation to solve the problem. Experiments show that our proposed method achieves high performance in completely heterogeneous scenarios where other approaches fail.