Can vision language models learn intuitive physics from interaction?
This addresses the problem of poor physical world understanding in AI models for researchers, showing incremental progress by highlighting limitations in generalization despite interaction-based training.
The study investigated whether vision language models can learn intuitive physics through interaction, finding that while reinforcement learning from interaction improves within-task performance, it fails to produce models with generalizable physical intuitions across related tasks.
Pre-trained vision language models do not have good intuitions about the physical world. Recent work has shown that supervised fine-tuning can improve model performance on simple physical tasks. However, fine-tuned models do not appear to learn robust physical rules that can generalize to new contexts. Based on research in cognitive science, we hypothesize that models need to interact with an environment to properly learn its physical dynamics. We train models that learn through interaction with the environment using reinforcement learning. While learning from interaction allows models to improve their within-task performance, it fails to produce models with generalizable physical intuitions. We find that models trained on one task do not reliably generalize to related tasks, even if the tasks share visual statistics and physical principles, and regardless of whether the models are trained through interaction.