Online Fine-Tuning of Pretrained Controllers for Autonomous Driving via Real-Time Recurrent RL
This addresses the challenge of adapting autonomous driving systems to real-world changes, though it appears incremental by combining existing methods like RTRRL with recurrent networks.
The paper tackled the problem of pretrained policies degrading in performance when autonomous systems encounter environmental changes, and showed that using Real-Time Recurrent Reinforcement Learning (RTRRL) to fine-tune these policies improves performance in simulated and real-world driving tasks.
Deploying pretrained policies in real-world applications presents substantial challenges that fundamentally limit the practical applicability of learning-based control systems. When autonomous systems encounter environmental changes in system dynamics, sensor drift, or task objectives, fixed policies rapidly degrade in performance. We show that employing Real-Time Recurrent Reinforcement Learning (RTRRL), a biologically plausible algorithm for online adaptation, can effectively fine-tune a pretrained policy to improve autonomous agents' performance on driving tasks. We further show that RTRRL synergizes with a recent biologically inspired recurrent network model, the Liquid-Resistance Liquid-Capacitance RNN. We demonstrate the effectiveness of this closed-loop approach in a simulated CarRacing environment and in a real-world line-following task with a RoboRacer car equipped with an event camera.