Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulation
This addresses the challenge of data efficiency in robotic manipulation, where data collection is expensive and imperfect data is often discarded, offering an incremental improvement over existing imitation learning methods.
The paper tackled the problem of leveraging imperfect data from task failures for robotic manipulation by introducing a Self-Supervised Data Filtering framework (SSDF) that computes quality scores for failed trajectory segments, expanding the training dataset and improving success rates in experiments on the ManiSkill2 benchmark and real-world tasks.
Improving data utilization, especially for imperfect data from task failures, is crucial for robotic manipulation due to the challenging, time-consuming, and expensive data collection process in the real world. Current imitation learning (IL) typically discards imperfect data, focusing solely on successful expert data. While reinforcement learning (RL) can learn from explorations and failures, the sim2real gap and its reliance on dense reward and online exploration make it difficult to apply effectively in real-world scenarios. In this work, we aim to conquer the challenge of leveraging imperfect data without the need for reward information to improve the model performance for robotic manipulation in an offline manner. Specifically, we introduce a Self-Supervised Data Filtering framework (SSDF) that combines expert and imperfect data to compute quality scores for failed trajectory segments. High-quality segments from the failed data are used to expand the training dataset. Then, the enhanced dataset can be used with any downstream policy learning method for robotic manipulation tasks. Extensive experiments on the ManiSkill2 benchmark built on the high-fidelity Sapien simulator and real-world robotic manipulation tasks using the Franka robot arm demonstrated that the SSDF can accurately expand the training dataset with high-quality imperfect data and improve the success rates for all robotic manipulation tasks.