Physics-Based Task Generation through Causal Sequence of Physical Interactions
This work addresses the problem of nuanced evaluation for physical reasoning agents, though it is incremental as it builds on existing simulation-based approaches.
The paper tackles the challenge of evaluating AI systems' physical reasoning by proposing a methodology for generating physics-based tasks using causal sequences of physical interactions, demonstrated with the Angry Birds game and evaluated using metrics like stability and solvability.
Performing tasks in a physical environment is a crucial yet challenging problem for AI systems operating in the real world. Physics simulation-based tasks are often employed to facilitate research that addresses this challenge. In this paper, first, we present a systematic approach for defining a physical scenario using a causal sequence of physical interactions between objects. Then, we propose a methodology for generating tasks in a physics-simulating environment using these defined scenarios as inputs. Our approach enables a better understanding of the granular mechanics required for solving physics-based tasks, thereby facilitating accurate evaluation of AI systems' physical reasoning capabilities. We demonstrate our proposed task generation methodology using the physics-based puzzle game Angry Birds and evaluate the generated tasks using a range of metrics, including physical stability, solvability using intended physical interactions, and accidental solvability using unintended solutions. We believe that the tasks generated using our proposed methodology can facilitate a nuanced evaluation of physical reasoning agents, thus paving the way for the development of agents for more sophisticated real-world applications.