ForeCite: Adapting Pre-Trained Language Models to Predict Future Citation Rates of Academic Papers
This work addresses the problem of automating research evaluation for scientists and institutions, though it is incremental as it builds on existing language models.
The paper tackled predicting future citation rates of academic papers by adapting pre-trained language models with a linear head for regression, achieving a test correlation of ρ=0.826 on a biomedical dataset, a 27-point improvement over previous state-of-the-art.
Predicting the future citation rates of academic papers is an important step toward the automation of research evaluation and the acceleration of scientific progress. We present $\textbf{ForeCite}$, a simple but powerful framework to append pre-trained causal language models with a linear head for average monthly citation rate prediction. Adapting transformers for regression tasks, ForeCite achieves a test correlation of $ρ= 0.826$ on a curated dataset of 900K+ biomedical papers published between 2000 and 2024, a 27-point improvement over the previous state-of-the-art. Comprehensive scaling-law analysis reveals consistent gains across model sizes and data volumes, while temporal holdout experiments confirm practical robustness. Gradient-based saliency heatmaps suggest a potentially undue reliance on titles and abstract texts. These results establish a new state-of-the-art in forecasting the long-term influence of academic research and lay the groundwork for the automated, high-fidelity evaluation of scientific contributions.