HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies
This addresses the problem of high-precision dexterous manipulation for robotics, particularly in real-world tasks like piano playing, though it is incremental as it builds on existing simulation-to-real transfer methods.
The paper tackles the challenge of achieving millimeter-scale precision in bimanual piano playing with a dexterous robot by introducing HandelBot, a framework that combines simulation policy refinement and residual reinforcement learning, resulting in successful performance across five songs with a 1.8x improvement over direct simulation deployment using only 30 minutes of physical data.
Mastering dexterous manipulation with multi-fingered hands has been a grand challenge in robotics for decades. Despite its potential, the difficulty of collecting high-quality data remains a primary bottleneck for high-precision tasks. While reinforcement learning and simulation-to-real-world transfer offer a promising alternative, the transferred policies often fail for tasks demanding millimeter-scale precision, such as bimanual piano playing. In this work, we introduce HandelBot, a framework that combines a simulation policy and rapid adaptation through a two-stage pipeline. Starting from a simulation-trained policy, we first apply a structured refinement stage to correct spatial alignments by adjusting lateral finger joints based on physical rollouts. Next, we use residual reinforcement learning to autonomously learn fine-grained corrective actions. Through extensive hardware experiments across five recognized songs, we demonstrate that HandelBot can successfully perform precise bimanual piano playing. Our system outperforms direct simulation deployment by a factor of 1.8x and requires only 30 minutes of physical interaction data.