Tackling GNARLy Problems: Graph Neural Algorithmic Reasoning Reimagined through Reinforcement Learning
This addresses the problem of learning algorithms for graph-based combinatorial problems, enabling broader applicability and improved performance, though it builds incrementally on existing NAR and RL methods.
The paper tackled the limitations of Neural Algorithmic Reasoning (NAR) by reframing algorithm learning as a Markov Decision Process using reinforcement learning, achieving high graph accuracy on CLRS-30 problems and matching or exceeding NAR on NP-hard problems without needing expert algorithms.
Neural Algorithmic Reasoning (NAR) is a paradigm that trains neural networks to execute classic algorithms by supervised learning. Despite its successes, important limitations remain: inability to construct valid solutions without post-processing and to reason about multiple correct ones, poor performance on combinatorial NP-hard problems, and inapplicability to problems for which strong algorithms are not yet known. To address these limitations, we reframe the problem of learning algorithm trajectories as a Markov Decision Process, which imposes structure on the solution construction procedure and unlocks the powerful tools of imitation and reinforcement learning (RL). We propose the GNARL framework, encompassing the methodology to translate problem formulations from NAR to RL and a learning architecture suitable for a wide range of graph-based problems. We achieve very high graph accuracy results on several CLRS-30 problems, performance matching or exceeding much narrower NAR approaches for NP-hard problems and, remarkably, applicability even when lacking an expert algorithm.