Incentive Mechanism Design for Distributed Ensemble Learning
This addresses the incentive gap in distributed ensemble learning, which is crucial for practical deployment but is incremental as it builds on existing algorithmic work.
The paper tackles the problem of incentivizing self-interested learners to participate in distributed ensemble learning by designing a mechanism that accounts for heterogeneous costs and data diversity, with numerical results on MNIST showing that a lower diversity level can lead to higher ensemble accuracy.
Distributed ensemble learning (DEL) involves training multiple models at distributed learners, and then combining their predictions to improve performance. Existing related studies focus on DEL algorithm design and optimization but ignore the important issue of incentives, without which self-interested learners may be unwilling to participate in DEL. We aim to fill this gap by presenting a first study on the incentive mechanism design for DEL. Our proposed mechanism specifies both the amount of training data and reward for learners with heterogeneous computation and communication costs. One design challenge is to have an accurate understanding regarding how learners' diversity (in terms of training data) affects the ensemble accuracy. To this end, we decompose the ensemble accuracy into a diversity-precision tradeoff to guide the mechanism design. Another challenge is that the mechanism design involves solving a mixed-integer program with a large search space. To this end, we propose an alternating algorithm that iteratively updates each learner's training data size and reward. We prove that under mild conditions, the algorithm converges. Numerical results using MNIST dataset show an interesting result: our proposed mechanism may prefer a lower level of learner diversity to achieve a higher ensemble accuracy.