With Good MT There is No Need For End-to-End: A Case for Translate-then-Summarize Cross-lingual Summarization
This work addresses the design choice for cross-lingual summarization systems, showing that incremental improvements in existing translation and summarization tasks can yield better results than end-to-end approaches, which is significant for NLP practitioners.
The authors tackled the problem of cross-lingual summarization by comparing pipeline (translate-then-summarize) and end-to-end system designs across 39 source languages into English, finding that the pipeline consistently outperforms end-to-end systems, with performance correlated to BLEU scores for weaker languages.
Recent work has suggested that end-to-end system designs for cross-lingual summarization are competitive solutions that perform on par or even better than traditional pipelined designs. A closer look at the evidence reveals that this intuition is based on the results of only a handful of languages or using underpowered pipeline baselines. In this work, we compare these two paradigms for cross-lingual summarization on 39 source languages into English and show that a simple \textit{translate-then-summarize} pipeline design consistently outperforms even an end-to-end system with access to enormous amounts of parallel data. For languages where our pipeline model does not perform well, we show that system performance is highly correlated with publicly distributed BLEU scores, allowing practitioners to establish the feasibility of a language pair a priori. Contrary to recent publication trends, our result suggests that the combination of individual progress of monolingual summarization and translation tasks offers better performance than an end-to-end system, suggesting that end-to-end designs should be considered with care.