Affordance-Based Hierarchical Reinforcement Learning for Quadruped Pedipulation
For quadruped robotics, this work addresses the challenge of autonomous object manipulation by eliminating the need for expert-designed trajectories, but it is an incremental step building on existing hierarchical RL and affordance concepts.
This paper proposes a three-level hierarchical reinforcement learning framework that uses pose and interaction-point affordances to enable quadruped robots to autonomously select interaction points and base poses for object manipulation, removing the need for pre-designed trajectories. The framework was trained in IsaacSim and validated in both simulation and real-world settings, successfully executing object manipulation tasks without human guidance.
The object manipulation capabilities of quadruped robots is an open research challenge. While previous studies have focused on low-level policy learning, task execution still relies on expert-designed high-level trajectories. Autonomous selection of both an affordable interaction point on the target object and an affordable robot base pose removes the need for pre-designed trajectories. This study proposes a three-level hierarchical reinforcement learning (RL) framework that utilizes pose affordances to guide the navigation policy, while the navigation policy drives the locomotion policy. In addition, the pedipulation policy is guided by interaction-point affordances, enabling object-centric pose alignment of the quadruped robot and effective end-effector manipulation planning. We train the proposed framework in the IsaacSim ecosystem and evaluate it in both simulation and real-world settings. We investigate the effectiveness of pose affordance across multiple scenarios in simulation while various object interaction tasks are validated on real-world setting forming an object-interaction dataset. The results show that the proposed framework can autonomously identify candidate poses based on their affordance and successfully execute object manipulation tasks in the real world without human guidance.