SIME: Enhancing Policy Self-Improvement with Modal-level Exploration
This work addresses the problem of enabling robots to enhance their capabilities autonomously through interaction, which is incremental as it builds on existing self-improvement methods.
The paper tackles the challenge of robotic self-improvement by introducing modal-level exploration and data selection to generate diverse interactions and select valuable data, demonstrating effective self-improvement in simulation and real-world experiments.
Self-improvement requires robotic systems to initially learn from human-provided data and then gradually enhance their capabilities through interaction with the environment. This is similar to how humans improve their skills through continuous practice. However, achieving effective self-improvement is challenging, primarily because robots tend to repeat their existing abilities during interactions, often failing to generate new, valuable data for learning. In this paper, we identify the key to successful self-improvement: modal-level exploration and data selection. By incorporating a modal-level exploration mechanism during policy execution, the robot can produce more diverse and multi-modal interactions. At the same time, we select the most valuable trials and high-quality segments from these interactions for learning. We successfully demonstrate effective robot self-improvement on both simulation benchmarks and real-world experiments. The capability for self-improvement will enable us to develop more robust and high-success-rate robotic control strategies at a lower cost. Our code and experiment scripts are available at https://ericjin2002.github.io/SIME/