A Hierarchical Language Model For Interpretable Graph Reasoning
This work addresses the challenge of applying LLMs to graph tasks for researchers and practitioners, though it appears incremental as it builds on existing LLM capabilities for graphs.
The paper tackles the problem of limited graph structure understanding in large language models (LLMs) when applied to large graphs, and introduces HLM-G, a hierarchical language model that enhances graph reasoning with high efficacy, efficiency, and robustness while reducing computational costs.
Large language models (LLMs) are being increasingly explored for graph tasks. Despite their remarkable success in text-based tasks, LLMs' capabilities in understanding explicit graph structures remain limited, particularly with large graphs. In this work, we introduce Hierarchical Language Model for Graph (HLM-G), which employs a two-block architecture to capture node-centric local information and interaction-centric global structure, effectively enhancing graph structure understanding abilities. The proposed scheme allows LLMs to address various graph queries with high efficacy, efficiency, and robustness, while reducing computational costs on large-scale graph tasks. Furthermore, we demonstrate the interpretability of our model using intrinsic attention weights and established explainers. Comprehensive evaluations across diverse graph reasoning and real-world tasks of node, link, and graph-levels highlight the superiority of our method, marking a significant advancement in the application of LLMs to graph understanding.