Q-CP: Learning Action Values for Cooperative Planning
This addresses the problem of efficient learning for multi-robot systems in unstructured scenarios, but it is incremental as it builds on existing methods like Q-learning and Monte-Carlo Tree Search.
The paper tackles the difficulty of learning effective robot behaviors in multi-robot systems due to high uncertainties and large state dimensionality by presenting Q-CP, a cooperative model-based reinforcement learning algorithm that uses action values to guide exploration and generate policies, showing effectiveness in reducing computational demand while achieving good performance in stochastic cooperative games.
Research on multi-robot systems has demonstrated promising results in manifold applications and domains. Still, efficiently learning an effective robot behaviors is very difficult, due to unstructured scenarios, high uncertainties, and large state dimensionality (e.g. hyper-redundant and groups of robot). To alleviate this problem, we present Q-CP a cooperative model-based reinforcement learning algorithm, which exploits action values to both (1) guide the exploration of the state space and (2) generate effective policies. Specifically, we exploit Q-learning to attack the curse-of-dimensionality in the iterations of a Monte-Carlo Tree Search. We implement and evaluate Q-CP on different stochastic cooperative (general-sum) games: (1) a simple cooperative navigation problem among 3 robots, (2) a cooperation scenario between a pair of KUKA YouBots performing hand-overs, and (3) a coordination task between two mobile robots entering a door. The obtained results show the effectiveness of Q-CP in the chosen applications, where action values drive the exploration and reduce the computational demand of the planning process while achieving good performance.