Refining Interactions: Enhancing Anisotropy in Graph Neural Networks with Language Semantics
This work addresses a bottleneck in graph-related tasks for researchers and practitioners by improving LLM-GNN integration, though it is incremental as it builds on anisotropic GNNs and LLM-based methods.
The paper tackles the problem of integrating Large Language Models (LLMs) with Graph Neural Networks (GNNs) for Text Attribute Graphs (TAGs), where existing methods treat structural information as general text, limiting effectiveness. It introduces LanSAGNN, which uses LLMs to extract semantic information for node pairs, significantly enhancing existing methods without increasing complexity and showing strong robustness.
The integration of Large Language Models (LLMs) with Graph Neural Networks (GNNs) has recently been explored to enhance the capabilities of Text Attribute Graphs (TAGs). Most existing methods feed textual descriptions of the graph structure or neighbouring nodes' text directly into LLMs. However, these approaches often cause LLMs to treat structural information simply as general contextual text, thus limiting their effectiveness in graph-related tasks. In this paper, we introduce LanSAGNN (Language Semantic Anisotropic Graph Neural Network), a framework that extends the concept of anisotropic GNNs to the natural language level. This model leverages LLMs to extract tailor-made semantic information for node pairs, effectively capturing the unique interactions within node relationships. In addition, we propose an efficient dual-layer LLMs finetuning architecture to better align LLMs' outputs with graph tasks. Experimental results demonstrate that LanSAGNN significantly enhances existing LLM-based methods without increasing complexity while also exhibiting strong robustness against interference.