On the Diversity and Limits of Human Explanations
This work addresses the problem of inconsistent explanation definitions in NLP for researchers and practitioners, but it is incremental as it synthesizes existing concepts from psychology.
The paper examines the diversity of human explanations in NLP, categorizing them into proximal mechanism, evidence, and procedure, and discusses implications for dataset collection and use, noting that explanations may require deeper understanding than predictions.
A growing effort in NLP aims to build datasets of human explanations. However, the term explanation encompasses a broad range of notions, each with different properties and ramifications. Our goal is to provide an overview of diverse types of explanations and human limitations, and discuss implications for collecting and using explanations in NLP. Inspired by prior work in psychology and cognitive sciences, we group existing human explanations in NLP into three categories: proximal mechanism, evidence, and procedure. These three types differ in nature and have implications for the resultant explanations. For instance, procedure is not considered explanations in psychology and connects with a rich body of work on learning from instructions. The diversity of explanations is further evidenced by proxy questions that are needed for annotators to interpret and answer open-ended why questions. Finally, explanations may require different, often deeper, understandings than predictions, which casts doubt on whether humans can provide useful explanations in some tasks.