A Survey of Federated Evaluation in Federated Learning
This addresses the problem of evaluating models without compromising data privacy for researchers and practitioners in federated learning, but it is incremental as it surveys existing methods rather than introducing new ones.
The paper tackles the challenge of model evaluation in federated learning, where data privacy prevents centralized access, by providing the first comprehensive survey of existing federated evaluation methods and exploring their applications to improve FL performance.
In traditional machine learning, it is trivial to conduct model evaluation since all data samples are managed centrally by a server. However, model evaluation becomes a challenging problem in federated learning (FL), which is called federated evaluation in this work. This is because clients do not expose their original data to preserve data privacy. Federated evaluation plays a vital role in client selection, incentive mechanism design, malicious attack detection, etc. In this paper, we provide the first comprehensive survey of existing federated evaluation methods. Moreover, we explore various applications of federated evaluation for enhancing FL performance and finally present future research directions by envisioning some challenges.