A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms
This addresses the challenge of efficient and effective policy learning in reinforcement learning for AI applications, representing a novel method rather than an incremental improvement.
The paper tackles the problem of learning optimal policies in reinforcement learning by proposing a control-theoretic approach, resulting in significant improvements in solution quality, sample complexity, and running time over state-of-the-art methods.
We devise a control-theoretic reinforcement learning approach to support direct learning of the optimal policy. We establish various theoretical properties of our approach, such as convergence and optimality of our analog of the Bellman operator and Q-learning, a new control-policy-variable gradient theorem, and a specific gradient ascent algorithm based on this theorem within the context of a specific control-theoretic framework. We empirically evaluate the performance of our control theoretic approach on several classical reinforcement learning tasks, demonstrating significant improvements in solution quality, sample complexity, and running time of our approach over state-of-the-art methods.