LyEvO: Lyapunov-Guided Evolutionary Optimization for Safe and Robust Sim-to-Real Policy Learning
This work provides a practical criterion for assessing deployment readiness of controllers, which is a critical problem for engineers developing robotic systems.
This paper addresses the challenge of safe and robust sim-to-real policy learning by proposing LyEvO, a framework that combines constrained Evolutionary Optimization, Statistical Model Checking, and Lyapunov-based stability analysis. The method iteratively optimizes and verifies policies while expanding the stability region, demonstrating safe and robust sim-to-real transfer on Cartpole and 3D Quadrotor benchmarks.
Training controllers that are safe and robust in simulation, and systematically assessing their readiness for real-world deployment, remain key challenges in sim-to-real transfer. To address this, we propose LyEvO, a physics-grounded framework that combines constrained Evolutionary Optimization and Statistical Model Checking (SMC)-based verification with Lyapunov-based stability analysis. Leveraging prior knowledge of the system dynamics, LyEvO uses Lyapunov analysis to compute an initial candidate stability region. An iterative loop then uses operational scenarios drawn from this region to jointly optimize and statistically verify a policy, and subsequently expands the region's boundaries based on the verification outcome. This integrated procedure provides a practical criterion for assessing deployment readiness. We evaluate LyEvO on Cartpole and 3D Quadrotor benchmarks through extensive simulations and targeted real-world experiments, demonstrating safe and robust sim-to-real transfer.