Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation
This work addresses data quality issues in cross-lingual summarization for NLP researchers, though it is incremental as it builds on existing benchmarks and methods.
The authors tackled the problem of errors in cross-lingual summarization corpora by proposing ConvSumX, a new benchmark with improved annotation that explicitly considers source context, and found it to be more faithful to input text. They also introduced a 2-Step method that outperformed strong baselines on ConvSumX in evaluations.
Most existing cross-lingual summarization (CLS) work constructs CLS corpora by simply and directly translating pre-annotated summaries from one language to another, which can contain errors from both summarization and translation processes. To address this issue, we propose ConvSumX, a cross-lingual conversation summarization benchmark, through a new annotation schema that explicitly considers source input context. ConvSumX consists of 2 sub-tasks under different real-world scenarios, with each covering 3 language directions. We conduct thorough analysis on ConvSumX and 3 widely-used manually annotated CLS corpora and empirically find that ConvSumX is more faithful towards input text. Additionally, based on the same intuition, we propose a 2-Step method, which takes both conversation and summary as input to simulate human annotation process. Experimental results show that 2-Step method surpasses strong baselines on ConvSumX under both automatic and human evaluation. Analysis shows that both source input text and summary are crucial for modeling cross-lingual summaries.