Affordance Learning from Play for Sample-Efficient Policy Learning
This addresses the challenge of sample-efficient policy learning for robots in human-centered environments, representing a novel method for a known bottleneck.
The paper tackles the problem of enabling robots to understand object functionality for manipulation tasks by proposing a self-supervised visual affordance model from human play data, which results in policies that train 4x faster and generalize better to novel objects compared to baselines.
Robots operating in human-centered environments should have the ability to understand how objects function: what can be done with each object, where this interaction may occur, and how the object is used to achieve a goal. To this end, we propose a novel approach that extracts a self-supervised visual affordance model from human teleoperated play data and leverages it to enable efficient policy learning and motion planning. We combine model-based planning with model-free deep reinforcement learning (RL) to learn policies that favor the same object regions favored by people, while requiring minimal robot interactions with the environment. We evaluate our algorithm, Visual Affordance-guided Policy Optimization (VAPO), with both diverse simulation manipulation tasks and real world robot tidy-up experiments to demonstrate the effectiveness of our affordance-guided policies. We find that our policies train 4x faster than the baselines and generalize better to novel objects because our visual affordance model can anticipate their affordance regions.