Constrained Reinforcement Learning for Robotics via Scenario-Based Programming
This addresses safety-critical tasks in robotics where human safety and expensive hardware are at risk, representing an incremental advancement by integrating expert knowledge into existing constrained DRL methods.
The paper tackled the problem of ensuring safety and performance in deep reinforcement learning for robotics by incorporating domain-expert knowledge via scenario-based programming, resulting in dramatic improvements in safety and performance validated through simulation and actual platform experiments.
Deep reinforcement learning (DRL) has achieved groundbreaking successes in a wide variety of robotic applications. A natural consequence is the adoption of this paradigm for safety-critical tasks, where human safety and expensive hardware can be involved. In this context, it is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our technique exploits the scenario-based programming paradigm, which is designed to allow specifying such knowledge in a simple and intuitive way. We validated our method on the popular robotic mapless navigation problem, in simulation, and on the actual platform. Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.