Online Decision Deferral under Budget Constraints
This work addresses the challenge of reducing expert workload in sequential decision-making tasks with limited resources, though it is incremental in applying bandit methods to deferral problems.
The paper tackled the problem of adaptively deferring decisions to human experts under budget constraints in online settings with potential distribution shifts, proposing a contextual bandit framework that achieves strong performance on real-world datasets.
Machine Learning (ML) models are increasingly used to support or substitute decision making. In applications where skilled experts are a limited resource, it is crucial to reduce their burden and automate decisions when the performance of an ML model is at least of equal quality. However, models are often pre-trained and fixed, while tasks arrive sequentially and their distribution may shift. In that case, the respective performance of the decision makers may change, and the deferral algorithm must remain adaptive. We propose a contextual bandit model of this online decision making problem. Our framework includes budget constraints and different types of partial feedback models. Beyond the theoretical guarantees of our algorithm, we propose efficient extensions that achieve remarkable performance on real-world datasets.