A New Dataset for Natural Language Inference from Code-mixed Conversations
This addresses the problem of limited resources for code-mixed NLI, which is incremental as it introduces a new dataset but does not propose a novel method.
The authors tackled the lack of datasets for natural language inference in code-mixed Hindi-English by creating the first such dataset using Bollywood movie snippets as premises and crowd-sourced hypotheses, resulting in 400 premises and 2240 hypotheses with evaluation using an mBERT-based pipeline.
Natural Language Inference (NLI) is the task of inferring the logical relationship, typically entailment or contradiction, between a premise and hypothesis. Code-mixing is the use of more than one language in the same conversation or utterance, and is prevalent in multilingual communities all over the world. In this paper, we present the first dataset for code-mixed NLI, in which both the premises and hypotheses are in code-mixed Hindi-English. We use data from Hindi movies (Bollywood) as premises, and crowd-source hypotheses from Hindi-English bilinguals. We conduct a pilot annotation study and describe the final annotation protocol based on observations from the pilot. Currently, the data collected consists of 400 premises in the form of code-mixed conversation snippets and 2240 code-mixed hypotheses. We conduct an extensive analysis to infer the linguistic phenomena commonly observed in the dataset obtained. We evaluate the dataset using a standard mBERT-based pipeline for NLI and report results.