Clustered Federated Learning via Embedding Distributions
This addresses data heterogeneity issues in federated learning for distributed environments like data protection scenarios, representing an incremental improvement over existing clustered FL methods.
The paper tackles the problem of non-IID data vulnerability in federated learning by proposing EMD-CFL, a one-shot clustering method using Earth Mover's distance in embedding space, which demonstrates superior clustering performance compared to 16 baselines on challenging datasets.
Federated learning (FL) is a widely used framework for machine learning in distributed data environments where clients hold data that cannot be easily centralised, such as for data protection reasons. FL, however, is known to be vulnerable to non-IID data. Clustered FL addresses this issue by finding more homogeneous clusters of clients. We propose a novel one-shot clustering method, EMD-CFL, using the Earth Mover's distance (EMD) between data distributions in embedding space. We theoretically motivate the use of EMDs using results from the domain adaptation literature and demonstrate empirically superior clustering performance in extensive comparisons against 16 baselines and on a range of challenging datasets.