FedBR: Improving Federated Learning on Heterogeneous Data via Local Learning Bias Reduction
This addresses the challenge of data heterogeneity in federated learning for privacy-preserving machine learning applications, representing an incremental improvement with a novel unified approach.
The paper tackles the problem of slow and unstable convergence in federated learning due to heterogeneous data by proposing FedBR, a novel algorithm that reduces local learning bias on features and classifiers, which consistently outperforms other state-of-the-art FL methods in experiments.
Federated Learning (FL) is a way for machines to learn from data that is kept locally, in order to protect the privacy of clients. This is typically done using local SGD, which helps to improve communication efficiency. However, such a scheme is currently constrained by slow and unstable convergence due to the variety of data on different clients' devices. In this work, we identify three under-explored phenomena of biased local learning that may explain these challenges caused by local updates in supervised FL. As a remedy, we propose FedBR, a novel unified algorithm that reduces the local learning bias on features and classifiers to tackle these challenges. FedBR has two components. The first component helps to reduce bias in local classifiers by balancing the output of the models. The second component helps to learn local features that are similar to global features, but different from those learned from other data sources. We conducted several experiments to test \algopt and found that it consistently outperforms other SOTA FL methods. Both of its components also individually show performance gains. Our code is available at https://github.com/lins-lab/fedbr.