Graph networks as learnable physics engines for inference and control
This work addresses the challenge of building machines with human-like physical understanding, offering a new framework for inference and control in dynamical systems, though it is incremental in advancing existing graph-based methods.
The paper tackled the problem of modeling complex physical systems for prediction and control by introducing graph network-based models that implement object- and relation-centric representations, achieving accurate predictions and strong generalization across eight varied physical systems.
Understanding and interacting with everyday physical scenes requires rich knowledge about the structure of the world, represented either implicitly in a value or policy function, or explicitly in a transition model. Here we introduce a new class of learnable models--based on graph networks--which implement an inductive bias for object- and relation-centric representations of complex, dynamical systems. Our results show that as a forward model, our approach supports accurate predictions from real and simulated data, and surprisingly strong and efficient generalization, across eight distinct physical systems which we varied parametrically and structurally. We also found that our inference model can perform system identification. Our models are also differentiable, and support online planning via gradient-based trajectory optimization, as well as offline policy optimization. Our framework offers new opportunities for harnessing and exploiting rich knowledge about the world, and takes a key step toward building machines with more human-like representations of the world.