Can Transformers Reason About Effects of Actions?
This work addresses the fundamental problem of enabling transformers to perform action-effect reasoning, a core challenge in knowledge representation and common sense AI for the broader AI community.
This paper investigates whether transformers can reason about the effects of actions by creating QA datasets across four action domains. The study evaluates their ability to learn reasoning within these domains and transfer that learning from a generic domain to others.
A recent work has shown that transformers are able to "reason" with facts and rules in a limited setting where the rules are natural language expressions of conjunctions of conditions implying a conclusion. Since this suggests that transformers may be used for reasoning with knowledge given in natural language, we do a rigorous evaluation of this with respect to a common form of knowledge and its corresponding reasoning -- the reasoning about effects of actions. Reasoning about action and change has been a top focus in the knowledge representation subfield of AI from the early days of AI and more recently it has been a highlight aspect in common sense question answering. We consider four action domains (Blocks World, Logistics, Dock-Worker-Robots and a Generic Domain) in natural language and create QA datasets that involve reasoning about the effects of actions in these domains. We investigate the ability of transformers to (a) learn to reason in these domains and (b) transfer that learning from the generic domains to the other domains.