Efficient and Effective Single-Document Summarizations and A Word-Embedding Measurement of Quality
This work addresses the need for efficient summarization tools in real-time applications, offering incremental improvements in algorithm performance and a new evaluation method.
The paper tackles the problem of generating effective single-document summaries under real-time constraints by proposing algorithms that use softplus-enhanced keyword rankings and topic clustering, achieving the best ROUGE recall scores on DUC-02. It also introduces WESM, a word-embedding-based measure, showing high comparability with ROUGE as an alternative quality metric.
Our task is to generate an effective summary for a given document with specific realtime requirements. We use the softplus function to enhance keyword rankings to favor important sentences, based on which we present a number of summarization algorithms using various keyword extraction and topic clustering methods. We show that our algorithms meet the realtime requirements and yield the best ROUGE recall scores on DUC-02 over all previously-known algorithms. We show that our algorithms meet the realtime requirements and yield the best ROUGE recall scores on DUC-02 over all previously-known algorithms. To evaluate the quality of summaries without human-generated benchmarks, we define a measure called WESM based on word-embedding using Word Mover's Distance. We show that the orderings of the ROUGE and WESM scores of our algorithms are highly comparable, suggesting that WESM may serve as a viable alternative for measuring the quality of a summary.