A Second Look on BASS -- Boosting Abstractive Summarization with Unified Semantic Graphs -- A Replication Study
This study addresses the replicability problem in NLP research, highlighting issues for researchers working with advanced summarization frameworks.
The authors conducted a replication study of the BASS framework for abstractive summarization, finding performance discrepancies compared to the original work and identifying challenges in replicating key components.
We present a detailed replication study of the BASS framework, an abstractive summarization system based on the notion of Unified Semantic Graphs. Our investigation includes challenges in replicating key components and an ablation study to systematically isolate error sources rooted in replicating novel components. Our findings reveal discrepancies in performance compared to the original work. We highlight the significance of paying careful attention even to reasonably omitted details for replicating advanced frameworks like BASS, and emphasize key practices for writing replicable papers.