Expertise Trees Resolve Knowledge Limitations in Collective Decision-Making
This addresses decision-making limitations for systems relying on expert advice, particularly in handling minority cases, though it appears incremental as it builds on existing modeling and algorithmic approaches.
The paper tackled the problem of sub-optimal or discriminatory decisions in collective decision-making due to varying expert knowledge across problem instances, by modeling expertise as a partition of the problem space and introducing expertise trees, which empirically improved performance on problems where existing methods failed.
Experts advising decision-makers are likely to display expertise which varies as a function of the problem instance. In practice, this may lead to sub-optimal or discriminatory decisions against minority cases. In this work we model such changes in depth and breadth of knowledge as a partitioning of the problem space into regions of differing expertise. We provide here new algorithms that explicitly consider and adapt to the relationship between problem instances and experts' knowledge. We first propose and highlight the drawbacks of a naive approach based on nearest neighbor queries. To address these drawbacks we then introduce a novel algorithm - expertise trees - that constructs decision trees enabling the learner to select appropriate models. We provide theoretical insights and empirically validate the improved performance of our novel approach on a range of problems for which existing methods proved to be inadequate.