AdaFed: Fair Federated Learning via Adaptive Common Descent Direction
This addresses fairness issues in federated learning for edge devices, though it appears incremental as it builds on existing fair FL methods.
The paper tackles the problem of unfair model training in federated learning by proposing AdaFed, which finds an adaptive common descent direction to ensure all clients' losses decrease, with faster rates for those with higher losses, and demonstrates that it outperforms state-of-the-art methods on federated datasets.
Federated learning (FL) is a promising technology via which some edge devices/clients collaboratively train a machine learning model orchestrated by a server. Learning an unfair model is known as a critical problem in federated learning, where the trained model may unfairly advantage or disadvantage some of the devices. To tackle this problem, in this work, we propose AdaFed. The goal of AdaFed is to find an updating direction for the server along which (i) all the clients' loss functions are decreasing; and (ii) more importantly, the loss functions for the clients with larger values decrease with a higher rate. AdaFed adaptively tunes this common direction based on the values of local gradients and loss functions. We validate the effectiveness of AdaFed on a suite of federated datasets, and demonstrate that AdaFed outperforms state-of-the-art fair FL methods.