Flexible Neural Representation for Physics Prediction
This work addresses the need for flexible physics prediction in computer vision, robotics, and cognitive science, offering a novel approach that is not incremental but introduces a new representation and architecture.
The paper tackled the problem of predicting physical dynamics for diverse 3D objects, including rigid shapes and deformable materials, by proposing a hierarchical particle-based representation and a neural network called HRN, which accurately handles complex collisions and deformations, scaling to large scenes and generating plausible long-term predictions.
Humans have a remarkable capacity to understand the physical dynamics of objects in their environment, flexibly capturing complex structures and interactions at multiple levels of detail. Inspired by this ability, we propose a hierarchical particle-based object representation that covers a wide variety of types of three-dimensional objects, including both arbitrary rigid geometrical shapes and deformable materials. We then describe the Hierarchical Relation Network (HRN), an end-to-end differentiable neural network based on hierarchical graph convolution, that learns to predict physical dynamics in this representation. Compared to other neural network baselines, the HRN accurately handles complex collisions and nonrigid deformations, generating plausible dynamics predictions at long time scales in novel settings, and scaling to large scene configurations. These results demonstrate an architecture with the potential to form the basis of next-generation physics predictors for use in computer vision, robotics, and quantitative cognitive science.