Discrete linear-complexity reinforcement learning in continuous action spaces for Q-learning algorithms
This work addresses a bottleneck in reinforcement learning for continuous control, offering an incremental improvement over existing discretization approaches.
The authors tackled the challenge of applying Q-learning to continuous action spaces by proposing a discretization method that avoids exponential dimensionality growth, resulting in a linear-complexity algorithm called Finite Step Q-Learning (FSQ).
In this article, we sketch an algorithm that extends the Q-learning algorithms to the continuous action space domain. Our method is based on the discretization of the action space. Despite the commonly used discretization methods, our method does not increase the discretized problem dimensionality exponentially. We will show that our proposed method is linear in complexity when the discretization is employed. The variant of the Q-learning algorithm presented in this work, labeled as Finite Step Q-Learning (FSQ), can be deployed to both shallow and deep neural network architectures.