A deep Natural Language Inference predictor without language-specific training data
This work addresses the challenge of cross-lingual NLI for languages lacking annotated data, though it is incremental as it builds on existing knowledge distillation techniques.
The paper tackles the problem of natural language inference (NLI) in a target language without language-specific training data by using knowledge distillation with a pre-trained model and a generic translation dataset, achieving results that outperform machine translation-based methods on datasets like Stanford NLI and Multi-Genre NLI.
In this paper we present a technique of NLP to tackle the problem of inference relation (NLI) between pairs of sentences in a target language of choice without a language-specific training dataset. We exploit a generic translation dataset, manually translated, along with two instances of the same pre-trained model - the first to generate sentence embeddings for the source language, and the second fine-tuned over the target language to mimic the first. This technique is known as Knowledge Distillation. The model has been evaluated over machine translated Stanford NLI test dataset, machine translated Multi-Genre NLI test dataset, and manually translated RTE3-ITA test dataset. We also test the proposed architecture over different tasks to empirically demonstrate the generality of the NLI task. The model has been evaluated over the native Italian ABSITA dataset, on the tasks of Sentiment Analysis, Aspect-Based Sentiment Analysis, and Topic Recognition. We emphasise the generality and exploitability of the Knowledge Distillation technique that outperforms other methodologies based on machine translation, even though the former was not directly trained on the data it was tested over.