Can Fair Federated Learning reduce the need for Personalisation?
This addresses the challenge of client participation in federated learning by reducing accuracy disparities, though it appears incremental as it builds on existing personalisation and fairness methods.
The paper tackles the problem of federated learning models underperforming on local client data, which disincentivizes participation, by evaluating Fair FL algorithms and proposing Personalisation-aware Federated Learning (PaFL). Results show PaFL reduces the number of underperforming clients by 50% in a language task and avoids doubling it in an image task.
Federated Learning (FL) enables training ML models on edge clients without sharing data. However, the federated model's performance on local data varies, disincentivising the participation of clients who benefit little from FL. Fair FL reduces accuracy disparity by focusing on clients with higher losses while personalisation locally fine-tunes the model. Personalisation provides a participation incentive when an FL model underperforms relative to one trained locally. For situations where the federated model provides a lower accuracy than a model trained entirely locally by a client, personalisation improves the accuracy of the pre-trained federated weights to be similar to or exceed those of the local client model. This paper evaluates two Fair FL (FFL) algorithms as starting points for personalisation. Our results show that FFL provides no benefit to relative performance in a language task and may double the number of underperforming clients for an image task. Instead, we propose Personalisation-aware Federated Learning (PaFL) as a paradigm that pre-emptively uses personalisation losses during training. Our technique shows a 50% reduction in the number of underperforming clients for the language task while lowering the number of underperforming clients in the image task instead of doubling it. Thus, evidence indicates that it may allow a broader set of devices to benefit from FL and represents a promising avenue for future experimentation and theoretical analysis.