Probing Physics Knowledge Using Tools from Developmental Psychology
This work addresses the challenge of assessing physics understanding in AI agents, which is incremental as it adapts an existing psychological method to a new domain.
The authors tackled the problem of evaluating intuitive physics knowledge in artificial learning systems by applying the violation of expectations (VOE) method from developmental psychology, and they introduced probe datasets to test a baseline deep learning system, showing that VOE may be a useful tool for future research.
In order to build agents with a rich understanding of their environment, one key objective is to endow them with a grasp of intuitive physics; an ability to reason about three-dimensional objects, their dynamic interactions, and responses to forces. While some work on this problem has taken the approach of building in components such as ready-made physics engines, other research aims to extract general physical concepts directly from sensory data. In the latter case, one challenge that arises is evaluating the learning system. Research on intuitive physics knowledge in children has long employed a violation of expectations (VOE) method to assess children's mastery of specific physical concepts. We take the novel step of applying this method to artificial learning systems. In addition to introducing the VOE technique, we describe a set of probe datasets inspired by classic test stimuli from developmental psychology. We test a baseline deep learning system on this battery, as well as on a physics learning dataset ("IntPhys") recently posed by another research group. Our results show how the VOE technique may provide a useful tool for tracking physics knowledge in future research.