CAIMAN: Causal Action Influence Detection for Sample-efficient Loco-manipulation
This addresses the challenge of sample-efficient learning for versatile robot manipulation in unstructured environments, representing an incremental improvement over existing methods.
The paper tackles the problem of enabling legged robots to perform non-prehensile loco-manipulation, such as object pushing, by introducing CAIMAN, a reinforcement learning framework that uses causal action influence as intrinsic motivation, resulting in superior sample efficiency and successful real-world transfer without fine-tuning.
Enabling legged robots to perform non-prehensile loco-manipulation is crucial for enhancing their versatility. Learning behaviors such as whole-body object pushing often requires sophisticated planning strategies or extensive task-specific reward shaping, especially in unstructured environments. In this work, we present CAIMAN, a practical reinforcement learning framework that encourages the agent to gain control over other entities in the environment. CAIMAN leverages causal action influence as an intrinsic motivation objective, allowing legged robots to efficiently acquire object pushing skills even under sparse task rewards. We employ a hierarchical control strategy, combining a low-level locomotion module with a high-level policy that generates task-relevant velocity commands and is trained to maximize the intrinsic reward. To estimate causal action influence, we learn the dynamics of the environment by integrating a kinematic prior with data collected during training.We empirically demonstrate CAIMAN's superior sample efficiency and adaptability to diverse scenarios in simulation, as well as its successful transfer to real-world systems without further fine-tuning.