Think about it! Improving defeasible reasoning by first modeling the question scenario
This addresses the challenge of enhancing reasoning capabilities in AI systems for tasks requiring nuanced inference, though it is incremental as it builds on existing cognitive science insights.
The paper tackles the problem of improving defeasible reasoning in neural models by having them first model the question scenario, achieving a new state-of-the-art on three datasets.
Defeasible reasoning is the mode of reasoning where conclusions can be overturned by taking into account new evidence. Existing cognitive science literature on defeasible reasoning suggests that a person forms a mental model of the problem scenario before answering questions. Our research goal asks whether neural models can similarly benefit from envisioning the question scenario before answering a defeasible query. Our approach is, given a question, to have a model first create a graph of relevant influences, and then leverage that graph as an additional input when answering the question. Our system, CURIOUS, achieves a new state-of-the-art on three different defeasible reasoning datasets. This result is significant as it illustrates that performance can be improved by guiding a system to "think about" a question and explicitly model the scenario, rather than answering reflexively. Code, data, and pre-trained models are located at https://github.com/madaan/thinkaboutit.