Neural Lyapunov Function Approximation with Self-Supervised Reinforcement Learning
This work addresses a complex problem in control theory for robotics, offering an incremental improvement over existing neural approximation methods.
The paper tackles the challenge of deriving control Lyapunov functions for nonlinear systems by introducing a sample-efficient neural approximation method that uses self-supervised reinforcement learning to improve training data generation, achieving faster convergence and higher accuracy compared to state-of-the-art baselines on robotic tasks.
Control Lyapunov functions are traditionally used to design a controller which ensures convergence to a desired state, yet deriving these functions for nonlinear systems remains a complex challenge. This paper presents a novel, sample-efficient method for neural approximation of nonlinear Lyapunov functions, leveraging self-supervised Reinforcement Learning (RL) to enhance training data generation, particularly for inaccurately represented regions of the state space. The proposed approach employs a data-driven World Model to train Lyapunov functions from off-policy trajectories. The method is validated on both standard and goal-conditioned robotic tasks, demonstrating faster convergence and higher approximation accuracy compared to the state-of-the-art neural Lyapunov approximation baseline. The code is available at: https://github.com/CAV-Research-Lab/SACLA.git