Reinforcement Learning with Fast Stabilization in Linear Dynamical Systems
This work addresses the challenge of fast stabilization in reinforcement learning for control tasks, offering a significant improvement over existing methods.
The paper tackles the problem of stabilizing unknown linear dynamical systems in model-based reinforcement learning to prevent system blow-ups, achieving a regret bound of $ ilde{\mathcal{O}}(\sqrt{T})$ with polynomial dependence on dimensions, which is an exponential improvement over prior methods.
In this work, we study model-based reinforcement learning (RL) in unknown stabilizable linear dynamical systems. When learning a dynamical system, one needs to stabilize the unknown dynamics in order to avoid system blow-ups. We propose an algorithm that certifies fast stabilization of the underlying system by effectively exploring the environment with an improved exploration strategy. We show that the proposed algorithm attains $\tilde{\mathcal{O}}(\sqrt{T})$ regret after $T$ time steps of agent-environment interaction. We also show that the regret of the proposed algorithm has only a polynomial dependence in the problem dimensions, which gives an exponential improvement over the prior methods. Our improved exploration method is simple, yet efficient, and it combines a sophisticated exploration policy in RL with an isotropic exploration strategy to achieve fast stabilization and improved regret. We empirically demonstrate that the proposed algorithm outperforms other popular methods in several adaptive control tasks.