Modular Multi-Objective Deep Reinforcement Learning with Decision Values
This addresses the need for multi-objective decision-making in areas like robotics and games, but it is incremental as it builds on existing DQN methods.
The paper tackles the problem of applying Deep Q-Networks to multi-objective environments by proposing a modular architecture with separate DQNs and decision values for scalarization, enabling controllable sub-behaviors and post-learning priority changes while maintaining performance in a 2D game-like simulator.
In this work we present a method for using Deep Q-Networks (DQNs) in multi-objective environments. Deep Q-Networks provide remarkable performance in single objective problems learning from high-level visual state representations. However, in many scenarios (e.g in robotics, games), the agent needs to pursue multiple objectives simultaneously. We propose an architecture in which separate DQNs are used to control the agent's behaviour with respect to particular objectives. In this architecture we introduce decision values to improve the scalarization of multiple DQNs into a single action. Our architecture enables the decomposition of the agent's behaviour into controllable and replaceable sub-behaviours learned by distinct modules. Moreover, it allows to change the priorities of particular objectives post-learning, while preserving the overall performance of the agent. To evaluate our solution we used a game-like simulator in which an agent - provided with high-level visual input - pursues multiple objectives in a 2D world.