Incentives for Federated Learning: a Hypothesis Elicitation Approach
This addresses the challenge of ensuring trustworthy participation from self-interested users in federated learning systems, which is incremental as it adapts information elicitation methods to a new context.
The paper tackles the problem of incentivizing truthful reporting of local models in federated learning by proposing a scoring rule framework based on hypothesis elicitation, and verifies its effectiveness on MNIST and CIFAR-10 datasets, showing that low-quality reports lead to decreasing rewards.
Federated learning provides a promising paradigm for collecting machine learning models from distributed data sources without compromising users' data privacy. The success of a credible federated learning system builds on the assumption that the decentralized and self-interested users will be willing to participate to contribute their local models in a trustworthy way. However, without proper incentives, users might simply opt out the contribution cycle, or will be mis-incentivized to contribute spam/false information. This paper introduces solutions to incentivize truthful reporting of a local, user-side machine learning model for federated learning. Our results build on the literature of information elicitation, but focus on the questions of eliciting hypothesis (rather than eliciting human predictions). We provide a scoring rule based framework that incentivizes truthful reporting of local hypotheses at a Bayesian Nash Equilibrium. We study the market implementation, accuracy as well as robustness properties of our proposed solution too. We verify the effectiveness of our methods using MNIST and CIFAR-10 datasets. Particularly we show that by reporting low-quality hypotheses, users will receive decreasing scores (rewards, or payments).