STAGE: Simplified Text-Attributed Graph Embeddings Using Pre-trained LLMs
This work addresses the need for simpler and more efficient methods in graph learning for researchers and practitioners, though it appears incremental as it builds on existing LLM and GNN approaches.
The paper tackled the problem of enhancing node features in Graph Neural Networks for Text-Attributed Graphs by proposing STAGE, a method that uses pre-trained Large-Language Models to generate embeddings, achieving competitive results on node classification benchmarks with simpler implementation than state-of-the-art techniques.
We present Simplified Text-Attributed Graph Embeddings (STAGE), a straightforward yet effective method for enhancing node features in Graph Neural Network (GNN) models that encode Text-Attributed Graphs (TAGs). Our approach leverages Large-Language Models (LLMs) to generate embeddings for textual attributes. STAGE achieves competitive results on various node classification benchmarks while also maintaining a simplicity in implementation relative to current state-of-the-art (SoTA) techniques. We show that utilizing pre-trained LLMs as embedding generators provides robust features for ensemble GNN training, enabling pipelines that are simpler than current SoTA approaches which require multiple expensive training and prompting stages. We also implement diffusion-pattern GNNs in an effort to make this pipeline scalable to graphs beyond academic benchmarks.