Cross-lingual Pseudo-Projected Expectation Regularization for Weakly Supervised Learning
This addresses the problem of limited labeled data for natural language processing in low-resource languages, offering a novel approach for cross-lingual transfer that is incremental but impactful.
The paper tackles cross-lingual weakly supervised learning by transferring knowledge from resource-rich languages via bitext, proposing a method that projects model expectations instead of labels to transfer uncertainty. It achieves F1 scores of 64% and 60% on Chinese-English and German-English NER datasets without labeled data, matching supervised performance with thousands of labeled sentences and setting new state-of-the-art results when combined with labeled examples.
We consider a multilingual weakly supervised learning scenario where knowledge from annotated corpora in a resource-rich language is transferred via bitext to guide the learning in other languages. Past approaches project labels across bitext and use them as features or gold labels for training. We propose a new method that projects model expectations rather than labels, which facilities transfer of model uncertainty across language boundaries. We encode expectations as constraints and train a discriminative CRF model using Generalized Expectation Criteria (Mann and McCallum, 2010). Evaluated on standard Chinese-English and German-English NER datasets, our method demonstrates F1 scores of 64% and 60% when no labeled data is used. Attaining the same accuracy with supervised CRFs requires 12k and 1.5k labeled sentences. Furthermore, when combined with labeled examples, our method yields significant improvements over state-of-the-art supervised methods, achieving best reported numbers to date on Chinese OntoNotes and German CoNLL-03 datasets.