Evaluating and Improving Interactions with Hazy Oracles
This work addresses the issue of unreliable human input in AI systems, offering a general framework that is incremental but applicable across domains.
The paper tackles the problem of AI systems treating human input as flawless by formalizing deferred inference, where the system defers decisions to disambiguate ambiguous human input, and introduces the Deferred Error Volume (DEV) metric to evaluate trade-offs between error reduction and human effort. It demonstrates this approach on video object tracking and referring expression comprehension, reducing error by up to 48% without modifying the underlying model.
Many AI systems integrate sensor inputs, world knowledge, and human-provided information to perform inference. While such systems often treat the human input as flawless, humans are better thought of as hazy oracles whose input may be ambiguous or outside of the AI system's understanding. In such situations it makes sense for the AI system to defer its inference while it disambiguates the human-provided information by, for example, asking the human to rephrase the query. Though this approach has been considered in the past, current work is typically limited to application-specific methods and non-standardized human experiments. We instead introduce and formalize a general notion of deferred inference. Using this formulation, we then propose a novel evaluation centered around the Deferred Error Volume (DEV) metric, which explicitly considers the tradeoff between error reduction and the additional human effort required to achieve it. We demonstrate this new formalization and an innovative deferred inference method on the disparate tasks of Single-Target Video Object Tracking and Referring Expression Comprehension, ultimately reducing error by up to 48% without any change to the underlying model or its parameters.