ESPRIT: Explaining Solutions to Physical Reasoning Tasks
This addresses the challenge of enabling AI systems to reason about physics for improved generalization, though it appears incremental as it builds on existing data-to-text methods.
The paper tackles the problem of neural networks lacking qualitative physical reasoning by proposing ESPRIT, a framework that generates interpretable natural language descriptions of physical events, with human evaluations showing it produces crucial fine-grained details and high coverage of physical concepts.
Neural networks lack the ability to reason about qualitative physics and so cannot generalize to scenarios and tasks unseen during training. We propose ESPRIT, a framework for commonsense reasoning about qualitative physics in natural language that generates interpretable descriptions of physical events. We use a two-step approach of first identifying the pivotal physical events in an environment and then generating natural language descriptions of those events using a data-to-text approach. Our framework learns to generate explanations of how the physical simulation will causally evolve so that an agent or a human can easily reason about a solution using those interpretable descriptions. Human evaluations indicate that ESPRIT produces crucial fine-grained details and has high coverage of physical concepts compared to even human annotations. Dataset, code and documentation are available at https://github.com/salesforce/esprit.