Internal and External Impacts of Natural Language Processing Papers
This work provides insights into how NLP research influences academia and society, highlighting gaps in attention to ethical issues, but it is incremental as it builds on existing citation and impact analysis methods.
The study analyzed the impacts of NLP research from 1979 to 2024, finding that language modeling has the widest influence both academically and externally, while topics like ethics and bias receive more policy attention than academic citations.
We investigate the impacts of NLP research published in top-tier conferences (i.e., ACL, EMNLP, and NAACL) from 1979 to 2024. By analyzing citations from research articles and external sources such as patents, media, and policy documents, we examine how different NLP topics are consumed both within the academic community and by the broader public. Our findings reveal that language modeling has the widest internal and external influence, while linguistic foundations have lower impacts. We also observe that internal and external impacts generally align, but topics like ethics, bias, and fairness show significant attention in policy documents with much fewer academic citations. Additionally, external domains exhibit distinct preferences, with patents focusing on practical NLP applications and media and policy documents engaging more with the societal implications of NLP models.