Online Regret Bounds for Undiscounted Continuous Reinforcement Learning
This work addresses the challenge of efficient online learning in continuous RL settings, representing an incremental advance in regret analysis.
The paper tackles the problem of deriving sublinear regret bounds for undiscounted reinforcement learning in continuous state spaces, achieving theoretical guarantees under assumptions of Holder continuity for rewards and transitions.
We derive sublinear regret bounds for undiscounted reinforcement learning in continuous state space. The proposed algorithm combines state aggregation with the use of upper confidence bounds for implementing optimism in the face of uncertainty. Beside the existence of an optimal policy which satisfies the Poisson equation, the only assumptions made are Holder continuity of rewards and transition probabilities.