Measuring and Modeling Physical Intrinsic Motivation
This work addresses how humans are motivated by interesting physical dynamics, with incremental improvements in modeling human responses.
The study formalized physical intrinsic motivation by collecting human ratings of interesting physics scenarios and modeling them with various hypotheses, finding that adversarial reward based on prediction loss best predicts responses, and combining it with collision count improves prediction, indicating humans seek high information gain and physical activity.
Humans are interactive agents driven to seek out situations with interesting physical dynamics. Here we formalize the functional form of physical intrinsic motivation. We first collect ratings of how interesting humans find a variety of physics scenarios. We then model human interestingness responses by implementing various hypotheses of intrinsic motivation including models that rely on simple scene features to models that depend on forward physics prediction. We find that the single best predictor of human responses is adversarial reward, a model derived from physical prediction loss. We also find that simple scene feature models do not generalize their prediction of human responses across all scenarios. Finally, linearly combining the adversarial model with the number of collisions in a scene leads to the greatest improvement in predictivity of human responses, suggesting humans are driven towards scenarios that result in high information gain and physical activity.