Equitable Federated Learning with Activation Clustering
This addresses algorithmic bias against specific groups in federated learning, offering a domain-specific solution that is incremental in nature.
The paper tackles bias in federated learning due to client heterogeneity by proposing an equitable clustering-based framework that uses activation vectors for similarity and a client weighing mechanism, achieving an O(1/√K) convergence rate to an ε-stationary solution and demonstrating reduced bias among client clusters.
Federated learning is a prominent distributed learning paradigm that incorporates collaboration among diverse clients, promotes data locality, and thus ensures privacy. These clients have their own technological, cultural, and other biases in the process of data generation. However, the present standard often ignores this bias/heterogeneity, perpetuating bias against certain groups rather than mitigating it. In response to this concern, we propose an equitable clustering-based framework where the clients are categorized/clustered based on how similar they are to each other. We propose a unique way to construct the similarity matrix that uses activation vectors. Furthermore, we propose a client weighing mechanism to ensure that each cluster receives equal importance and establish $O(1/\sqrt{K})$ rate of convergence to reach an $ε-$stationary solution. We assess the effectiveness of our proposed strategy against common baselines, demonstrating its efficacy in terms of reducing the bias existing amongst various client clusters and consequently ameliorating algorithmic bias against specific groups.