Can ChatGPT Make Explanatory Inferences? Benchmarks for Abductive Reasoning
This addresses the problem of assessing explanatory reasoning in AI for researchers and developers, though it is incremental as it focuses on benchmarking an existing model.
The paper introduces benchmarks to evaluate AI's ability to perform explanatory inference and tests ChatGPT, finding it capable of creative and evaluative inferences in verbal and visual domains, rebutting claims that such models lack explanation and creativity.
Explanatory inference is the creation and evaluation of hypotheses that provide explanations, and is sometimes known as abduction or abductive inference. Generative AI is a new set of artificial intelligence models based on novel algorithms for generating text, images, and sounds. This paper proposes a set of benchmarks for assessing the ability of AI programs to perform explanatory inference, and uses them to determine the extent to which ChatGPT, a leading generative AI model, is capable of making explanatory inferences. Tests on the benchmarks reveal that ChatGPT performs creative and evaluative inferences in many domains, although it is limited to verbal and visual modalities. Claims that ChatGPT and similar models are incapable of explanation, understanding, causal reasoning, meaning, and creativity are rebutted.