Hybrid Decision Making via Conformal VLM-generated Guidance
For human-AI teams in high-stakes domains like medical diagnosis, ConfGuide reduces cognitive load while maintaining decision quality.
ConfGuide improves hybrid decision making by generating succinct, targeted textual guidance that caps false negative rates via conformal risk control, showing promise on a multi-label medical diagnosis task.
Building on recent advances in AI, hybrid decision making (HDM) holds the promise of improving human decision quality and reducing cognitive load. We work in the context of learning to guide (LtG), a recently proposed HDM framework in which the human is always responsible for the final decision: rather than suggesting decisions, in LtG the AI supplies (textual) guidance useful for facilitating decision making. One limiting factor of existing approaches is that their guidance compounds information about all possible outcomes, and as a result it can be difficult to digest. We address this issue by introducing ConfGuide, a novel LtG approach that generates more succinct and targeted guidance. To this end, it employs conformal risk control to select a set of outcomes, ensuring a cap on the false negative rate. We demonstrate our approach on a real-world multi-label medical diagnosis task. Our empirical evaluation highlights the promise of ConfGuide.