Citation Classification for Behavioral Analysis of a Scientific Field
This work addresses the need for large-scale analysis of citation behavior in scientific fields, particularly for researchers and scholars in natural language processing, though it is incremental as it builds on existing citation analysis methods with a new dataset.
The authors tackled the problem of understanding citation behavior in scientific fields by performing the largest behavioral study to date in natural language processing, analyzing nearly 2,000 annotated citations and labeling the entire ACL Reference Corpus to reveal patterns in citation framing and uptake, such as authors being sensitive to discourse structure and how citation roles predict citation counts.
Citations are an important indicator of the state of a scientific field, reflecting how authors frame their work, and influencing uptake by future scholars. However, our understanding of citation behavior has been limited to small-scale manual citation analysis. We perform the largest behavioral study of citations to date, analyzing how citations are both framed and taken up by scholars in one entire field: natural language processing. We introduce a new dataset of nearly 2,000 citations annotated for function and centrality, and use it to develop a state-of-the-art classifier and label the entire ACL Reference Corpus. We then study how citations are framed by authors and use both papers and online traces to track how citations are followed by readers. We demonstrate that authors are sensitive to discourse structure and publication venue when citing, that online readers follow temporal links to previous and future work rather than methodological links, and that how a paper cites related work is predictive of its citation count. Finally, we use changes in citation roles to show that the field of NLP is undergoing a significant increase in consensus.