FlowControl: Optical Flow Based Visual Servoing
This provides a robust and easy-to-set-up method for robot programming from demonstrations, addressing a bottleneck in robotics.
The paper tackles one-shot imitation for robot manipulation by using learning-based optical flow to perform real-time visual servoing from a single demonstration video, achieving practical results on tasks requiring precise motions and generalization.
One-shot imitation is the vision of robot programming from a single demonstration, rather than by tedious construction of computer code. We present a practical method for realizing one-shot imitation for manipulation tasks, exploiting modern learning-based optical flow to perform real-time visual servoing. Our approach, which we call FlowControl, continuously tracks a demonstration video, using a specified foreground mask to attend to an object of interest. Using RGB-D observations, FlowControl requires no 3D object models, and is easy to set up. FlowControl inherits great robustness to visual appearance from decades of work in optical flow. We exhibit FlowControl on a range of problems, including ones requiring very precise motions, and ones requiring the ability to generalize.