Unsupervised Natural Language Inference Using PHL Triplet Generation
This addresses the challenge of training NLI models without human-annotated data, which is beneficial for researchers and practitioners in NLP facing data scarcity, though it is incremental as it builds on existing unsupervised methods.
The paper tackles the problem of unsupervised Natural Language Inference (NLI) by proposing a procedural data generation method using sentence transformations to create PHL triplets, achieving accuracies up to 66.75% and showing that fine-tuning with minimal human-annotated data yields a 12.2% accuracy improvement.
Transformer-based models achieve impressive performance on numerous Natural Language Inference (NLI) benchmarks when trained on respective training datasets. However, in certain cases, training samples may not be available or collecting them could be time-consuming and resource-intensive. In this work, we address the above challenge and present an explorative study on unsupervised NLI, a paradigm in which no human-annotated training samples are available. We investigate it under three settings: PH, P, and NPH that differ in the extent of unlabeled data available for learning. As a solution, we propose a procedural data generation approach that leverages a set of sentence transformations to collect PHL (Premise, Hypothesis, Label) triplets for training NLI models, bypassing the need for human-annotated training data. Comprehensive experiments with several NLI datasets show that the proposed approach results in accuracies of up to 66.75%, 65.9%, 65.39% in PH, P, and NPH settings respectively, outperforming all existing unsupervised baselines. Furthermore, fine-tuning our model with as little as ~0.1% of the human-annotated training dataset (500 instances) leads to 12.2% higher accuracy than the model trained from scratch on the same 500 instances. Supported by this superior performance, we conclude with a recommendation for collecting high-quality task-specific data.