GammaZero: Learning To Guide POMDP Belief Space Search With Graph Representations
This addresses scalability issues in POMDP planning for domains like robotics or AI, offering a novel method for generalization but is incremental in improving existing graph-based approaches.
The paper tackled the problem of scaling planning in Partially Observable Markov Decision Processes (POMDPs) by introducing an action-centric graph representation framework, achieving zero-shot generalization to problems 2-4 times larger than training instances while maintaining solution quality with reduced search.
We introduce an action-centric graph representation framework for learning to guide planning in Partially Observable Markov Decision Processes (POMDPs). Unlike existing approaches that require domain-specific neural architectures and struggle with scalability, GammaZero leverages a unified graph-based belief representation that enables generalization across problem sizes within a domain. Our key insight is that belief states can be systematically transformed into action-centric graphs where structural patterns learned on small problems transfer to larger instances. We employ a graph neural network with a decoder architecture to learn value functions and policies from expert demonstrations on computationally tractable problems, then apply these learned heuristics to guide Monte Carlo tree search on larger problems. Experimental results on standard POMDP benchmarks demonstrate that GammaZero achieves comparable performance to BetaZero when trained and tested on the same-sized problems, while uniquely enabling zero-shot generalization to problems 2-4 times larger than those seen during training, maintaining solution quality with reduced search requirements.