UnNatural Language Inference
This research identifies a fundamental limitation in the syntactic understanding of current NLU models, suggesting they do not process language in a human-like manner, which is critical for researchers developing more robust and human-cognition-aligned AI.
This paper investigates the understanding of syntax in large pre-trained Transformer models by testing their invariance to random word-order permutations in Natural Language Inference (NLI) tasks. It finds that state-of-the-art NLI models assign the same labels to permuted examples as to original ones, with 98.7% of MNLI examples retaining their gold label after at least one permutation.
Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to know humanlike syntax, at least to some extent. We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i.e. they are largely invariant to random word-order permutations. This behavior notably differs from that of humans; we struggle with ungrammatical sentences. To measure the severity of this issue, we propose a suite of metrics and investigate which properties of particular permutations lead models to be word-order invariant. In the MNLI dataset, for example, we find almost all (98.7%) examples contain at least one permutation which elicits the gold label. Models are sometimes even able to assign gold labels to permutations that they originally failed to predict correctly. We provide a comprehensive empirical evaluation of this phenomenon, and further show that this issue exists for both Transformers and pre-Transformer RNN / ConvNet based encoders, as well as across multiple languages (English and Mandarin Chinese). Our code and data are available at https://github.com/facebookresearch/unlu.