Improving Fairness for Data Valuation in Horizontal Federated Learning
This work addresses fairness issues in incentivizing data owner participation in federated learning, which is crucial for sustaining collaborative ML systems, though it is incremental as it builds on prior Shapley value methods.
The paper tackles the problem of unfairness in data valuation for horizontal federated learning, where existing Federated Shapley value can assign different values to data owners with identical data, and proposes a completed federated Shapley value that improves fairness, as verified by theoretical and empirical results.
Federated learning is an emerging decentralized machine learning scheme that allows multiple data owners to work collaboratively while ensuring data privacy. The success of federated learning depends largely on the participation of data owners. To sustain and encourage data owners' participation, it is crucial to fairly evaluate the quality of the data provided by the data owners and reward them correspondingly. Federated Shapley value, recently proposed by Wang et al. [Federated Learning, 2020], is a measure for data value under the framework of federated learning that satisfies many desired properties for data valuation. However, there are still factors of potential unfairness in the design of federated Shapley value because two data owners with the same local data may not receive the same evaluation. We propose a new measure called completed federated Shapley value to improve the fairness of federated Shapley value. The design depends on completing a matrix consisting of all the possible contributions by different subsets of the data owners. It is shown under mild conditions that this matrix is approximately low-rank by leveraging concepts and tools from optimization. Both theoretical analysis and empirical evaluation verify that the proposed measure does improve fairness in many circumstances.