Eliminating Domain Bias for Federated Learning in Representation Space
This work addresses the challenge of domain bias in federated learning for privacy-preserving collaborative systems, offering a solution to improve generalization and personalization, though it appears incremental as it builds on existing FL methods.
The paper tackles the problem of domain bias in federated learning under statistically heterogeneous scenarios, which causes representation degeneration, and proposes the Domain Bias Eliminator (DBE) framework to address it, showing that DBE-equipped methods outperform ten state-of-the-art personalized FL methods by a large margin in experiments on four datasets.
Recently, federated learning (FL) is popular for its privacy-preserving and collaborative learning abilities. However, under statistically heterogeneous scenarios, we observe that biased data domains on clients cause a representation bias phenomenon and further degenerate generic representations during local training, i.e., the representation degeneration phenomenon. To address these issues, we propose a general framework Domain Bias Eliminator (DBE) for FL. Our theoretical analysis reveals that DBE can promote bi-directional knowledge transfer between server and client, as it reduces the domain discrepancy between server and client in representation space. Besides, extensive experiments on four datasets show that DBE can greatly improve existing FL methods in both generalization and personalization abilities. The DBE-equipped FL method can outperform ten state-of-the-art personalized FL methods by a large margin. Our code is public at https://github.com/TsingZ0/DBE.