SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer
This addresses the safety challenge in sim-to-real transfer for robotics, offering a practical solution with provable guarantees, though it builds incrementally on existing domain randomization methods.
The paper tackles the problem of safely deploying reinforcement learning policies from simulators to real-world robots by proposing SPiDR, a scalable algorithm that ensures safety despite the sim-to-real gap while maintaining strong performance, as demonstrated through experiments on benchmarks and real-world platforms.
Deploying reinforcement learning (RL) safely in the real world is challenging, as policies trained in simulators must face the inevitable sim-to-real gap. Robust safe RL techniques are provably safe, however difficult to scale, while domain randomization is more practical yet prone to unsafe behaviors. We address this gap by proposing SPiDR, short for Sim-to-real via Pessimistic Domain Randomization -- a scalable algorithm with provable guarantees for safe sim-to-real transfer. SPiDR uses domain randomization to incorporate the uncertainty about the sim-to-real gap into the safety constraints, making it versatile and highly compatible with existing training pipelines. Through extensive experiments on sim-to-sim benchmarks and two distinct real-world robotic platforms, we demonstrate that SPiDR effectively ensures safety despite the sim-to-real gap while maintaining strong performance.