OTTR: Off-Road Trajectory Tracking using Reinforcement Learning
This addresses the sim-to-real gap for off-road vehicle control, enabling more accurate trajectory tracking in complex environments with minimal real-world data, though it is incremental in improving existing RL approaches.
The paper tackles the off-road trajectory tracking problem by proposing a novel Reinforcement Learning algorithm that uses supervised learning to adapt a baseline policy with limited real-world data, achieving a 30% and 50% reduction in cross track error for two vehicles compared to standard methods.
In this work, we present a novel Reinforcement Learning (RL) algorithm for the off-road trajectory tracking problem. Off-road environments involve varying terrain types and elevations, and it is difficult to model the interaction dynamics of specific off-road vehicles with such a diverse and complex environment. Standard RL policies trained on a simulator will fail to operate in such challenging real-world settings. Instead of using a naive domain randomization approach, we propose an innovative supervised-learning based approach for overcoming the sim-to-real gap problem. Our approach efficiently exploits the limited real-world data available to adapt the baseline RL policy obtained using a simple kinematics simulator. This avoids the need for modeling the diverse and complex interaction of the vehicle with off-road environments. We evaluate the performance of the proposed algorithm using two different off-road vehicles, Warthog and Moose. Compared to the standard ILQR approach, our proposed approach achieves a 30% and 50% reduction in cross track error in Warthog and Moose, respectively, by utilizing only 30 minutes of real-world driving data.