SECite: Analyzing and Summarizing Citations in Software Engineering Literature
This work addresses the need for more objective literature reviews in software engineering by providing a tool to analyze external citation feedback, though it is incremental as it builds on existing sentiment analysis and summarization techniques.
The researchers tackled the problem of evaluating scholarly impact by analyzing citation sentiments in software engineering literature, developing SECite, a semi-automated pipeline that uses NLP and unsupervised machine learning to classify citations as positive or negative and generate sentiment-specific summaries, revealing patterns in community perception and alignment with authors' self-presentations.
Identifying the strengths and limitations of a research paper is a core component of any literature review. However, traditional summaries reflect only the authors' self-presented perspective. Analyzing how other researchers discuss and cite the paper can offer a deeper, more practical understanding of its contributions and shortcomings. In this research, we introduce SECite, a novel approach for evaluating scholarly impact through sentiment analysis of citation contexts. We develop a semi-automated pipeline to extract citations referencing nine research papers and apply advanced natural language processing (NLP) techniques with unsupervised machine learning to classify these citation statements as positive or negative. Beyond sentiment classification, we use generative AI to produce sentiment-specific summaries that capture the strengths and limitations of each target paper, derived both from clustered citation groups and from the full text. Our findings reveal meaningful patterns in how the academic community perceives these works, highlighting areas of alignment and divergence between external citation feedback and the authors' own presentation. By integrating citation sentiment analysis with LLM-based summarization, this study provides a comprehensive framework for assessing scholarly contributions.