Realistic Zero-Shot Cross-Lingual Transfer in Legal Topic Classification
This addresses the problem of legal document classification across languages for practitioners, but it is incremental as it builds on existing datasets and methods.
The paper tackled zero-shot cross-lingual transfer in legal topic classification by creating a more realistic dataset without parallel documents, showing that translation-based methods outperform previous methods and that a bilingual teacher-student approach exceeds direct fine-tuning on target language data.
We consider zero-shot cross-lingual transfer in legal topic classification using the recent MultiEURLEX dataset. Since the original dataset contains parallel documents, which is unrealistic for zero-shot cross-lingual transfer, we develop a new version of the dataset without parallel documents. We use it to show that translation-based methods vastly outperform cross-lingual fine-tuning of multilingually pre-trained models, the best previous zero-shot transfer method for MultiEURLEX. We also develop a bilingual teacher-student zero-shot transfer approach, which exploits additional unlabeled documents of the target language and performs better than a model fine-tuned directly on labeled target language documents.