LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification
This work addresses the problem of understanding and optimizing LLM-GNN integration for graph representation learning, but it appears incremental as it builds on existing enhancer approaches with a new analysis and module.
The paper tackles the underexplored properties of using large language models (LLMs) as feature enhancers for graph neural networks (GNNs) by analyzing them with interchange interventions on a synthetic graph dataset, and it designs an optimization module that improves information transfer, validated across multiple datasets and models.
The use of large language models (LLMs) as feature enhancers to optimize node representations, which are then used as inputs for graph neural networks (GNNs), has shown significant potential in graph representation learning. However, the fundamental properties of this approach remain underexplored. To address this issue, we propose conducting a more in-depth analysis of this issue based on the interchange intervention method. First, we construct a synthetic graph dataset with controllable causal relationships, enabling precise manipulation of semantic relationships and causal modeling to provide data for analysis. Using this dataset, we conduct interchange interventions to examine the deeper properties of LLM enhancers and GNNs, uncovering their underlying logic and internal mechanisms. Building on the analytical results, we design a plug-and-play optimization module to improve the information transfer between LLM enhancers and GNNs. Experiments across multiple datasets and models validate the proposed module.