FedGA: Federated Learning with Gradient Alignment for Error Asymmetry Mitigation
This work addresses bias in federated learning models due to class imbalance, which is an incremental improvement over existing methods like FedAvg.
The paper tackles the problem of error asymmetry in federated learning caused by inter-client class imbalance, proposing FedGA which uses gradient alignment to mitigate bias and improve model convergence and accuracy, achieving higher F1 scores and accuracy margins on five benchmark datasets as the Dirichlet sampling factor increases.
Federated learning (FL) triggers intra-client and inter-client class imbalance, with the latter compared to the former leading to biased client updates and thus deteriorating the distributed models. Such a bias is exacerbated during the server aggregation phase and has yet to be effectively addressed by conventional re-balancing methods. To this end, different from the off-the-shelf label or loss-based approaches, we propose a gradient alignment (GA)-informed FL method, dubbed as FedGA, where the importance of error asymmetry (EA) in bias is observed and its linkage to the gradient of the loss to raw logits is explored. Concretely, GA, implemented by label calibration during the model backpropagation process, prevents catastrophic forgetting of rate and missing classes, hence boosting model convergence and accuracy. Experimental results on five benchmark datasets demonstrate that GA outperforms the pioneering counterpart FedAvg and its four variants in minimizing EA and updating bias, and accordingly yielding higher F1 score and accuracy margins when the Dirichlet distribution sampling factor $α$ increases. The code and more details are available at \url{https://anonymous.4open.science/r/FedGA-B052/README.md}.