Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames
This addresses the challenge of requiring numerous demonstrations for generalization in foundation models for physical world tasks, offering a method to reduce demonstration needs, though it is incremental as it builds on existing behavioral cloning techniques.
The paper tackles the problem of improving generalization in behavioral cloning for sequential decision-making by proposing that demonstrators be 'blindfolded' to some task information, forcing exploration. Experiments on robotic peg insertion and Procgen videogames show this approach generalizes better to unseen tasks, with theoretical analysis indicating generalization error scales with the square root of the information available to the demonstrator divided by the number of tasks.
Behavioral cloning is a simple yet effective technique for learning sequential decision-making from demonstrations. Recently, it has gained prominence as the core of foundation models for the physical world, where achieving generalization requires countless demonstrations of a multitude of tasks. Typically, a human expert with full information on the task demonstrates a (nearly) optimal behavior. In this paper, we propose to hide some of the task's information from the demonstrator. This ``blindfolded'' expert is compelled to employ non-trivial exploration to solve the task. We show that cloning the blindfolded expert generalizes better to unseen tasks than its fully-informed counterpart. We conduct experiments of real-world robot peg insertion tasks with (limited) human demonstrations, alongside videogames from the Procgen benchmark. Additionally, we support our findings with theoretical analysis, which confirms that the generalization error scales with $\sqrt{I/m}$, where $I$ measures the amount of task information available to the demonstrator, and $m$ is the number of demonstrated tasks. Both theory and practice indicate that cloning blindfolded experts generalizes better with fewer demonstrated tasks. Project page with videos and code: https://sites.google.com/view/blindfoldedexperts/home