IntPhys: A Framework and Benchmark for Visual Intuitive Physics Reasoning
This addresses the problem of enabling AI systems to understand basic physics for complex visual tasks, though it is incremental as it builds on existing intuitive physics research.
The authors introduced IntPhys, a benchmark for evaluating visual intuitive physics reasoning by distinguishing possible vs. impossible events in videos, and tested two deep neural networks trained unsupervisedly, finding they achieved competitive performance but with limitations compared to humans.
In order to reach human performance on complexvisual tasks, artificial systems need to incorporate a sig-nificant amount of understanding of the world in termsof macroscopic objects, movements, forces, etc. Inspiredby work on intuitive physics in infants, we propose anevaluation benchmark which diagnoses how much a givensystem understands about physics by testing whether itcan tell apart well matched videos of possible versusimpossible events constructed with a game engine. Thetest requires systems to compute a physical plausibilityscore over an entire video. It is free of bias and cantest a range of basic physical reasoning concepts. Wethen describe two Deep Neural Networks systems aimedat learning intuitive physics in an unsupervised way,using only physically possible videos. The systems aretrained with a future semantic mask prediction objectiveand tested on the possible versus impossible discrimi-nation task. The analysis of their results compared tohuman data gives novel insights in the potentials andlimitations of next frame prediction architectures.