CLIRFeb 26, 2024

Pre-training Cross-lingual Open Domain Question Answering with Large-scale Synthetic Supervision

arXiv:2402.16508v324 citationsh-index: 3EMNLP
Originality Incremental advance
AI Analysis

This addresses the problem of reducing annotation and resource needs for multilingual question answering, though it is incremental as it builds on existing self-supervised techniques.

The paper tackles cross-lingual open domain question answering by proposing a single encoder-decoder model trained with self-supervised synthetic supervision from Wikipedia links, outperforming comparable methods in supervised and zero-shot settings.

Cross-lingual open domain question answering (CLQA) is a complex problem, comprising cross-lingual retrieval from a multilingual knowledge base, followed by answer generation in the query language. Both steps are usually tackled by separate models, requiring substantial annotated datasets, and typically auxiliary resources, like machine translation systems to bridge between languages. In this paper, we show that CLQA can be addressed using a single encoder-decoder model. To effectively train this model, we propose a self-supervised method based on exploiting the cross-lingual link structure within Wikipedia. We demonstrate how linked Wikipedia pages can be used to synthesise supervisory signals for cross-lingual retrieval, through a form of cloze query, and generate more natural questions to supervise answer generation. Together, we show our approach, \texttt{CLASS}, outperforms comparable methods on both supervised and zero-shot language adaptation settings, including those using machine translation.

Code Implementations1 repo
Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes