From Node2Vec to GPT-based GraphRAG: scientific impact prediction across graph and language models
For researchers and funding agencies needing to predict citation impact under cold-start conditions, this work provides a comparative evaluation of graph and language models, though the findings are incremental.
The paper formulates scientific impact prediction as a cohort-normalized top-P% classification task and compares graph-based (Node2Vec) and LLM-based (GPT GraphRAG) approaches. The best supervised method achieves ~0.84-0.85 AUC, while GPT-based GraphRAG reaches 0.87 AUC but retrieval augmentation does not consistently outperform simpler LLM baselines.
Identifying which newly published scientific papers are likely to become highly cited is important for prioritizing research attention, supporting editorial decisions, and guiding the allocation of scientific resources, particularly under cold-start conditions where little direct evidence is available at publication time. In this work, we formulate impact prediction as a cohort-normalized top-P% classification task and compare graph-based and LLM-based approaches under a unified framework. We construct citation and textual-similarity graphs under temporal constraints and generate Node2Vec representations, either alone or combined with OpenAI text embeddings. The best supervised configuration combines directed citation graphs with textual embeddings, reaching approximately 0.84-0.85 AUC. We also evaluate a GPT-based GraphRAG setup, using GPT 5.5 and 5.4 Nano, in which graph neighborhoods are used as contextual evidence for prediction. Although the LLM-based approach achieves high performance, retrieved context does not consistently improve results; target-only prompts often perform as well as or better than GraphRAG prompts achieving the 0.87 mark. These findings indicate that structural and textual signals are complementary for supervised prediction, while retrieval augmentation must be carefully evaluated against simpler LLM baselines.