Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning
This addresses robot safety in unpredictable environments, offering a novel approach to reduce human interventions, though it appears incremental by building on existing foundation models and planning methods.
The paper tackles the problem of preventing safety-critical failures in robots during out-of-distribution scenarios by developing FORTRESS, a framework that uses multi-modal reasoning and planning to generate real-time fallback strategies, resulting in improved safety classification accuracy and planning success in simulations and hardware tests.
While foundation models offer promise toward improving robot safety in out-of-distribution (OOD) scenarios, how to effectively harness their generalist knowledge for real-time, dynamically feasible response remains a crucial problem. We present FORTRESS, a joint reasoning and planning framework that generates semantically safe fallback strategies to prevent safety-critical, OOD failures. At a low frequency under nominal operation, FORTRESS uses multi-modal foundation models to anticipate possible failure modes and identify safe fallback sets. When a runtime monitor triggers a fallback response, FORTRESS rapidly synthesizes plans to fallback goals while inferring and avoiding semantically unsafe regions in real time. By bridging open-world, multi-modal reasoning with dynamics-aware planning, we eliminate the need for hard-coded fallbacks and human safety interventions. FORTRESS outperforms on-the-fly prompting of slow reasoning models in safety classification accuracy on synthetic benchmarks and real-world ANYmal robot data, and further improves system safety and planning success in simulation and on quadrotor hardware for urban navigation. Website can be found at https://milanganai.github.io/fortress.