Learning Affordance Landscapes for Interaction Exploration in 3D Environments
This addresses the challenge of interaction exploration for embodied agents in human spaces, representing an incremental advance by combining reinforcement learning with affordance modeling for specific 3D environments.
The paper tackles the problem of enabling embodied agents to autonomously discover how to interact with objects in unfamiliar 3D environments, such as kitchens, by introducing a reinforcement learning approach that maximizes successful interactions and trains an affordance segmentation model, resulting in agents learning to use new environments intelligently and rapidly address downstream tasks like 'find a knife and put it in the drawer'.
Embodied agents operating in human spaces must be able to master how their environment works: what objects can the agent use, and how can it use them? We introduce a reinforcement learning approach for exploration for interaction, whereby an embodied agent autonomously discovers the affordance landscape of a new unmapped 3D environment (such as an unfamiliar kitchen). Given an egocentric RGB-D camera and a high-level action space, the agent is rewarded for maximizing successful interactions while simultaneously training an image-based affordance segmentation model. The former yields a policy for acting efficiently in new environments to prepare for downstream interaction tasks, while the latter yields a convolutional neural network that maps image regions to the likelihood they permit each action, densifying the rewards for exploration. We demonstrate our idea with AI2-iTHOR. The results show agents can learn how to use new home environments intelligently and that it prepares them to rapidly address various downstream tasks like "find a knife and put it in the drawer." Project page: http://vision.cs.utexas.edu/projects/interaction-exploration/