Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions
This addresses safety-critical decision-making in autonomous systems like robotics and finance, offering a novel framework for risk calibration.
The paper tackles the problem of making safe autonomous decisions using imperfect machine learning predictions, introducing Conformal Decision Theory to provide provable statistical guarantees of low risk without assumptions on the world model, with experiments in robot motion planning, stock trading, and manufacturing.
We introduce Conformal Decision Theory, a framework for producing safe autonomous decisions despite imperfect machine learning predictions. Examples of such decisions are ubiquitous, from robot planning algorithms that rely on pedestrian predictions, to calibrating autonomous manufacturing to exhibit high throughput and low error, to the choice of trusting a nominal policy versus switching to a safe backup policy at run-time. The decisions produced by our algorithms are safe in the sense that they come with provable statistical guarantees of having low risk without any assumptions on the world model whatsoever; the observations need not be I.I.D. and can even be adversarial. The theory extends results from conformal prediction to calibrate decisions directly, without requiring the construction of prediction sets. Experiments demonstrate the utility of our approach in robot motion planning around humans, automated stock trading, and robot manufacturing.