Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control
This addresses the challenge of whole-body control for humanoid robots, enabling remote operation in diverse environments, though it is incremental as it builds on existing RL and motion retargeting techniques.
The paper tackles the problem of achieving precise upper-body manipulation alongside robust lower-body locomotion in humanoid robots by decoupling control, using inverse kinematics and motion retargeting for manipulation and reinforcement learning for locomotion, and shows that their method significantly outperforms RL-based whole-body control in precise manipulation.
Humanoid robots require both robust lower-body locomotion and precise upper-body manipulation. While recent Reinforcement Learning (RL) approaches provide whole-body loco-manipulation policies, they lack precise manipulation with high DoF arms. In this paper, we propose decoupling upper-body control from locomotion, using inverse kinematics (IK) and motion retargeting for precise manipulation, while RL focuses on robust lower-body locomotion. We introduce PMP (Predictive Motion Priors), trained with Conditional Variational Autoencoder (CVAE) to effectively represent upper-body motions. The locomotion policy is trained conditioned on this upper-body motion representation, ensuring that the system remains robust with both manipulation and locomotion. We show that CVAE features are crucial for stability and robustness, and significantly outperforms RL-based whole-body control in precise manipulation. With precise upper-body motion and robust lower-body locomotion control, operators can remotely control the humanoid to walk around and explore different environments, while performing diverse manipulation tasks.