Conditional Visual Servoing for Multi-Step Tasks
This work addresses the problem of enabling robots to perform complex multi-step tasks using visual servoing without manual programming, though it is incremental as it builds on existing visual servoing methods.
The paper tackles the limitation of visual servoing to single demonstrations by proposing a modular approach called conditional servoing, which selects the next demonstration based on robot observations to handle multi-step tasks, achieving best results with a reprojection error selection function in simulation and real robot experiments.
Visual Servoing has been effectively used to move a robot into specific target locations or to track a recorded demonstration. It does not require manual programming, but it is typically limited to settings where one demonstration maps to one environment state. We propose a modular approach to extend visual servoing to scenarios with multiple demonstration sequences. We call this conditional servoing, as we choose the next demonstration conditioned on the observation of the robot. This method presents an appealing strategy to tackle multi-step problems, as individual demonstrations can be combined flexibly into a control policy. We propose different selection functions and compare them on a shape-sorting task in simulation. With the reprojection error yielding the best overall results, we implement this selection function on a real robot and show the efficacy of the proposed conditional servoing. For videos of our experiments, please check out our project page: https://lmb.informatik.uni-freiburg.de/projects/conditional_servoing/