Neural Task Representations as Weak Supervision for Model Agnostic Cross-Lingual Transfer
This addresses the high annotation costs and engineering effort for cross-lingual NLP, making it easier to deploy models in non-English languages, though it is incremental as it builds on existing transfer methods.
The paper tackles the problem of transferring neural models from English to other languages by proposing a framework that uses task representations as weak supervision, requiring only unlabeled parallel data and outperforming strong baselines while rivaling state-of-the-art results with less complexity and resources.
Natural language processing is heavily Anglo-centric, while the demand for models that work in languages other than English is greater than ever. Yet, the task of transferring a model from one language to another can be expensive in terms of annotation costs, engineering time and effort. In this paper, we present a general framework for easily and effectively transferring neural models from English to other languages. The framework, which relies on task representations as a form of weak supervision, is model and task agnostic, meaning that many existing neural architectures can be ported to other languages with minimal effort. The only requirement is unlabeled parallel data, and a loss defined over task representations. We evaluate our framework by transferring an English sentiment classifier to three different languages. On a battery of tests, we show that our models outperform a number of strong baselines and rival state-of-the-art results, which rely on more complex approaches and significantly more resources and data. Additionally, we find that the framework proposed in this paper is able to capture semantically rich and meaningful representations across languages, despite the lack of direct supervision.