Learning to Guide Human Experts via Personalized Large Language Models
This addresses the issue of human over-reliance on AI decisions in high-stakes domains like healthcare, though it is incremental as it builds on existing deferral frameworks.
The paper tackles the problem of anchoring bias in learning to defer by proposing learning to guide (LTG), a framework where the machine provides textual guidance instead of decisions, and presents promising preliminary results on a medical diagnosis task.
In learning to defer, a predictor identifies risky decisions and defers them to a human expert. One key issue with this setup is that the expert may end up over-relying on the machine's decisions, due to anchoring bias. At the same time, whenever the machine chooses the deferral option the expert has to take decisions entirely unassisted. As a remedy, we propose learning to guide (LTG), an alternative framework in which -- rather than suggesting ready-made decisions -- the machine provides guidance useful to guide decision-making, and the human is entirely responsible for coming up with a decision. We also introduce SLOG, an LTG implementation that leverages (a small amount of) human supervision to convert a generic large language model into a module capable of generating textual guidance, and present preliminary but promising results on a medical diagnosis task.