MADDM: Multi-Advisor Dynamic Binary Decision-Making by Maximizing the Utility
This addresses a crucial problem in applications like lending and crowd-sourcing by maximizing utility, but it appears incremental as it builds on existing decision-making models.
The paper tackles the problem of inferring ground truth from multiple imperfect advisors with costs, proposing a strategy for optimal advisor selection in sequential binary decision-making without prior knowledge, and shows it outperforms state-of-the-art models in experiments.
Being able to infer ground truth from the responses of multiple imperfect advisors is a problem of crucial importance in many decision-making applications, such as lending, trading, investment, and crowd-sourcing. In practice, however, gathering answers from a set of advisors has a cost. Therefore, finding an advisor selection strategy that retrieves a reliable answer and maximizes the overall utility is a challenging problem. To address this problem, we propose a novel strategy for optimally selecting a set of advisers in a sequential binary decision-making setting, where multiple decisions need to be made over time. Crucially, we assume no access to ground truth and no prior knowledge about the reliability of advisers. Specifically, our approach considers how to simultaneously (1) select advisors by balancing the advisors' costs and the value of making correct decisions, (2) learn the trustworthiness of advisers dynamically without prior information by asking multiple advisers, and (3) make optimal decisions without access to the ground truth, improving this over time. We evaluate our algorithm through several numerical experiments. The results show that our approach outperforms two other methods that combine state-of-the-art models.