Prompting Contrastive Explanations for Commonsense Reasoning Tasks
This addresses the issue of opaque reasoning in NLP models for researchers and practitioners, though it is incremental as it builds on existing prompting techniques.
The paper tackled the problem of generating human-interpretable evidence for commonsense reasoning tasks by using pretrained language models to produce contrastive explanations, which improved performance on two benchmarks compared to non-contrastive methods.
Many commonsense reasoning NLP tasks involve choosing between one or more possible answers to a question or prompt based on knowledge that is often implicit. Large pretrained language models (PLMs) can achieve near-human performance on such tasks, while providing little human-interpretable evidence of the underlying reasoning they use. In this work, we show how to use these same models to generate such evidence: inspired by the contrastive nature of human explanations, we use PLMs to complete explanation prompts which contrast alternatives according to the key attribute(s) required to justify the correct answer (for example, peanuts are usually salty while raisins are sweet). Conditioning model decisions on these explanations improves performance on two commonsense reasoning benchmarks, as compared to previous non-contrastive alternatives. These explanations are also judged by humans to be more relevant for solving the task, and facilitate a novel method to evaluate explanation faithfulfness.