Robust Federated Personalised Mean Estimation for the Gaussian Mixture Model
This work addresses robustness and personalization in federated learning for heterogeneous data, but it is incremental as it focuses on a simplified instantiation of the broader problem.
The paper tackles the problem of robust personalized mean estimation in federated learning with corrupted clients, where data follows a Gaussian mixture model, and presents an algorithm with error scaling almost linearly with the corruption ratio, supported by a matching lower bound up to a constant factor.
Federated learning with heterogeneous data and personalization has received significant recent attention. Separately, robustness to corrupted data in the context of federated learning has also been studied. In this paper we explore combining personalization for heterogeneous data with robustness, where a constant fraction of the clients are corrupted. Motivated by this broad problem, we formulate a simple instantiation which captures some of its difficulty. We focus on the specific problem of personalized mean estimation where the data is drawn from a Gaussian mixture model. We give an algorithm whose error depends almost linearly on the ratio of corrupted to uncorrupted samples, and show a lower bound with the same behavior, albeit with a gap of a constant factor.