FedCME: Client Matching and Classifier Exchanging to Handle Data Heterogeneity in Federated Learning
This addresses the problem of slow convergence and weakened performance in federated learning for applications with non-IID data, though it is incremental as it builds on existing client-based methods.
The paper tackles data heterogeneity in federated learning by proposing FedCME, a framework that matches clients with different data distributions and exchanges their classifiers during local training, which improves global model performance. Experimental results show FedCME outperforms FedAvg, FedProx, MOON, and FedRS on benchmarks like FMNIST and CIFAR10 under heterogeneous data conditions.
Data heterogeneity across clients is one of the key challenges in Federated Learning (FL), which may slow down the global model convergence and even weaken global model performance. Most existing approaches tackle the heterogeneity by constraining local model updates through reference to global information provided by the server. This can alleviate the performance degradation on the aggregated global model. Different from existing methods, we focus the information exchange between clients, which could also enhance the effectiveness of local training and lead to generate a high-performance global model. Concretely, we propose a novel FL framework named FedCME by client matching and classifier exchanging. In FedCME, clients with large differences in data distribution will be matched in pairs, and then the corresponding pair of clients will exchange their classifiers at the stage of local training in an intermediate moment. Since the local data determines the local model training direction, our method can correct update direction of classifiers and effectively alleviate local update divergence. Besides, we propose feature alignment to enhance the training of the feature extractor. Experimental results demonstrate that FedCME performs better than FedAvg, FedProx, MOON and FedRS on popular federated learning benchmarks including FMNIST and CIFAR10, in the case where data are heterogeneous.