Findings of the 2016 WMT Shared Task on Cross-lingual Pronoun Prediction
This work addresses a specific challenge in machine translation for linguists and NLP researchers, but it is incremental as it builds on existing shared task frameworks.
The paper tackled the problem of cross-lingual pronoun prediction as a classification task for English-French and English-German language pairs, with results showing that most submissions outperformed strong baselines and deep recurrent neural networks achieved the best performance.
We describe the design, the evaluation setup, and the results of the 2016 WMT shared task on cross-lingual pronoun prediction. This is a classification task in which participants are asked to provide predictions on what pronoun class label should replace a placeholder value in the target-language text, provided in lemmatised and PoS-tagged form. We provided four subtasks, for the English-French and English-German language pairs, in both directions. Eleven teams participated in the shared task; nine for the English-French subtask, five for French-English, nine for English-German, and six for German-English. Most of the submissions outperformed two strong language-model based baseline systems, with systems using deep recurrent neural networks outperforming those using other architectures for most language pairs.