Online Markov decision processes with policy iteration
This work addresses the challenge of adapting MDPs to dynamic environments for applications in reinforcement learning and control, representing an incremental improvement with theoretical guarantees.
The authors tackled the problem of online Markov decision processes with changing reward functions by proposing practical algorithms using policy iteration, achieving a sublinear regret bound and demonstrating efficiency in handling large or continuous state spaces through experiments.
The online Markov decision process (MDP) is a generalization of the classical Markov decision process that incorporates changing reward functions. In this paper, we propose practical online MDP algorithms with policy iteration and theoretically establish a sublinear regret bound. A notable advantage of the proposed algorithm is that it can be easily combined with function approximation, and thus large and possibly continuous state spaces can be efficiently handled. Through experiments, we demonstrate the usefulness of the proposed algorithm.