Configurable Markov Decision Processes
This work addresses a novel interaction paradigm for learning agents in real-world scenarios, though it appears incremental as it builds on existing MDP frameworks.
The authors tackled the problem of optimizing agent performance by configuring environmental parameters, introducing Configurable Markov Decision Processes (Conf-MDPs) and a Safe Policy-Model Iteration (SPMI) algorithm, with experimental results showing improved policy performance in two test problems.
In many real-world problems, there is the possibility to configure, to a limited extent, some environmental parameters to improve the performance of a learning agent. In this paper, we propose a novel framework, Configurable Markov Decision Processes (Conf-MDPs), to model this new type of interaction with the environment. Furthermore, we provide a new learning algorithm, Safe Policy-Model Iteration (SPMI), to jointly and adaptively optimize the policy and the environment configuration. After having introduced our approach and derived some theoretical results, we present the experimental evaluation in two explicative problems to show the benefits of the environment configurability on the performance of the learned policy.