SumQE: a BERT-based Summary Quality Estimation Model
This work addresses the need for automated quality estimation in summarization systems, offering a tool for developers and users, though it appears incremental as it builds on existing BERT and quality estimation approaches.
The authors tackled the problem of evaluating summary quality without human references by proposing SumQE, a BERT-based model that focuses on linguistic aspects, achieving very high correlations with human ratings.
We propose SumQE, a novel Quality Estimation model for summarization based on BERT. The model addresses linguistic quality aspects that are only indirectly captured by content-based approaches to summary evaluation, without involving comparison with human references. SumQE achieves very high correlations with human ratings, outperforming simpler models addressing these linguistic aspects. Predictions of the SumQE model can be used for system development, and to inform users of the quality of automatically produced summaries and other types of generated text.