Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks and Autoregressive Policy Decomposition
This addresses the challenge of handling relational problems in RL for domains with variable structures, offering a generalizable solution.
The authors tackled reinforcement learning in relational problems with variable state and action spaces by developing a domain-independent framework based on graph neural networks and autoregressive policy decomposition, demonstrating broad applicability and impressive zero-shot generalization across three distinct domains.
We focus on reinforcement learning (RL) in relational problems that are naturally defined in terms of objects, their relations, and object-centric actions. These problems are characterized by variable state and action spaces, and finding a fixed-length representation, required by most existing RL methods, is difficult, if not impossible. We present a deep RL framework based on graph neural networks and auto-regressive policy decomposition that naturally works with these problems and is completely domain-independent. We demonstrate the framework's broad applicability in three distinct domains and show impressive zero-shot generalization over different problem sizes.