On the impact of MDP design for Reinforcement Learning agents in Resource Management
This work addresses the impact of MDP design on RL agents for resource management, but it is incremental as it focuses on empirical comparisons of existing variations.
The paper tackles the problem of how MDP design choices affect reinforcement learning agent performance in resource management, finding that a compact state representation enables agent transfer across environments and outperforms specialized agents in 80% of scenarios without retraining.
The recent progress in Reinforcement Learning applications to Resource Management presents MDPs without a deeper analysis of the impacts of design decisions on agent performance. In this paper, we compare and contrast four different MDP variations, discussing their computational requirements and impacts on agent performance by means of an empirical analysis. We conclude by showing that, in our experiments, when using Multi-Layer Perceptrons as approximation function, a compact state representation allows transfer of agents between environments, and that transferred agents have good performance and outperform specialized agents in 80\% of the tested scenarios, even without retraining.