Less is More: Learning Graph Tasks with Just LLMs
This addresses the challenge of graph reasoning for AI applications, though it appears incremental by building on prior work on LLM graph reasoning.
The paper tackles the problem of enabling large language models (LLMs) to solve graph tasks without specialized graph encoders, showing that small LLMs can learn and generalize to new tasks and structures using instructive chain-of-thought training.
For large language models (LLMs), reasoning over graphs could help solve many problems. Prior work has tried to improve LLM graph reasoning by examining how best to serialize graphs as text and by combining GNNs and LLMs. However, the merits of such approaches remain unclear, so we empirically answer the following research questions: (1) Can LLMs learn to solve fundamental graph tasks without specialized graph encoding models?, (2) Can LLMs generalize learned solutions to unseen graph structures or tasks?, and (3) What are the merits of competing approaches to learn graph tasks? We show that even small LLMs can learn to solve graph tasks by training them with instructive chain-of-thought solutions, and this training generalizes, without specialized graph encoders, to new tasks and graph structures.