Learning to Play with Intrinsically-Motivated Self-Aware Agents
This work addresses the challenge of creating flexible autonomous agents that can self-supervise in complex novel physical environments, representing an incremental step in this domain.
The paper tackled the problem of generating novel structured behaviors in unstructured environments without clear extrinsic rewards by formalizing curiosity-driven intrinsic motivation using a neural network. The result was that the agent explored novel interactions, leading to complex behaviors like object gathering and improved performance on object dynamics prediction, detection, localization, and recognition tasks.
Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically naturalistic simulated environment in which an agent can move and interact with objects it sees, we propose a "world-model" network that learns to predict the dynamic consequences of the agent's actions. Simultaneously, we train a separate explicit "self-model" that allows the agent to track the error map of its own world-model, and then uses the self-model to adversarially challenge the developing world-model. We demonstrate that this policy causes the agent to explore novel and informative interactions with its environment, leading to the generation of a spectrum of complex behaviors, including ego-motion prediction, object attention, and object gathering. Moreover, the world-model that the agent learns supports improved performance on object dynamics prediction, detection, localization and recognition tasks. Taken together, our results are initial steps toward creating flexible autonomous agents that self-supervise in complex novel physical environments.