How Safe Am I Given What I See? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy
This work addresses the safety prediction problem for image-controlled autonomous systems, offering a novel approach to improve safety assurance in end-to-end learning, though it is incremental in building upon existing generative world models and conformal prediction methods.
The paper tackles the safety assurance challenge in end-to-end learning for autonomous systems by proposing a configurable family of learning pipelines based on generative world models to predict safety chances without relying on low-dimensional states, achieving statistical calibration guarantees through conformal prediction. It evaluates these pipelines on case studies of a racing car and a cartpole, demonstrating their effectiveness in providing calibrated safety predictions.
End-to-end learning has emerged as a major paradigm for developing autonomous systems. Unfortunately, with its performance and convenience comes an even greater challenge of safety assurance. A key factor of this challenge is the absence of the notion of a low-dimensional and interpretable dynamical state, around which traditional assurance methods revolve. Focusing on the online safety prediction problem, this paper proposes a configurable family of learning pipelines based on generative world models, which do not require low-dimensional states. To implement these pipelines, we overcome the challenges of learning safety-informed latent representations and missing safety labels under prediction-induced distribution shift. These pipelines come with statistical calibration guarantees on their safety chance predictions based on conformal prediction. We perform an extensive evaluation of the proposed learning pipelines on two case studies of image-controlled systems: a racing car and a cartpole.