Deep Visual Foresight for Planning Robot Motion
This addresses the challenge of scaling robot learning to many skills and environments by reducing reliance on costly human feedback, though it is incremental as it builds on existing model-based reinforcement learning approaches.
The paper tackles the problem of enabling robots to learn manipulation skills without human supervision by developing a method that combines deep action-conditioned video prediction with model-predictive control, resulting in a real robot successfully pushing objects and handling novel ones not seen in training.
A key challenge in scaling up robot learning to many skills and environments is removing the need for human supervision, so that robots can collect their own data and improve their own performance without being limited by the cost of requesting human feedback. Model-based reinforcement learning holds the promise of enabling an agent to learn to predict the effects of its actions, which could provide flexible predictive models for a wide range of tasks and environments, without detailed human supervision. We develop a method for combining deep action-conditioned video prediction models with model-predictive control that uses entirely unlabeled training data. Our approach does not require a calibrated camera, an instrumented training set-up, nor precise sensing and actuation. Our results show that our method enables a real robot to perform nonprehensile manipulation -- pushing objects -- and can handle novel objects not seen during training.