GMLM: Bridging Graph Neural Networks and Language Models for Heterophilic Node Classification
This work addresses a central problem in text-rich heterophilic graph learning for researchers and practitioners, representing an incremental advancement in fusion methods.
The paper tackled the challenge of integrating pre-trained language models with graph neural networks for heterophilic node classification, achieving state-of-the-art results on four out of five benchmarks, with improvements such as over 8% accuracy on the Texas dataset.
Integrating Pre-trained Language Models (PLMs) with Graph Neural Networks (GNNs) remains a central challenge in text-rich heterophilic graph learning. We propose a novel integration framework that enables effective fusion between powerful pre-trained text encoders and Relational Graph Convolutional Networks (R-GCNs). Our method enhances the alignment of textual and structural representations through a bidirectional fusion mechanism and contrastive node-level optimization. To evaluate the approach, we train two variants using different PLMs: Snowflake-Embed (state-of-the-art) and GTE-base, each paired with an R-GCN backbone. Experiments on five heterophilic benchmarks demonstrate that our integration method achieves state-of-the-art results on four datasets, surpassing existing GNN and large language model-based approaches. Notably, Snowflake-Embed + R-GCN improves accuracy on the Texas dataset by over 8\% and on Wisconsin by nearly 5\%. These results highlight the effectiveness of our fusion strategy for advancing text-rich graph representation learning.