Language-Grounded Decoupled Action Representation for Robotic Manipulation
This addresses the problem of robust and accurate action generation for novel tasks in robotic manipulation, representing an incremental improvement over existing methods.
The paper tackles the challenge of aligning high-level vision-language understanding with low-level action control in robotic manipulation by proposing the LaDA framework, which uses language to connect perception and control through interpretable action primitives, achieving strong performance and generalization in simulated and real-world experiments.
The heterogeneity between high-level vision-language understanding and low-level action control remains a fundamental challenge in robotic manipulation. Although recent methods have advanced task-specific action alignment, they often struggle to generate robust and accurate actions for novel or semantically related tasks. To address this, we propose the Language-Grounded Decoupled Action Representation (LaDA) framework, which leverages natural language as a semantic bridge to connect perception and control. LaDA introduces a fine-grained intermediate layer of three interpretable action primitives--translation, rotation, and gripper control--providing explicit semantic structure for low-level actions. It further employs a semantic-guided soft-label contrastive learning objective to align similar action primitives across tasks, enhancing generalization and motion consistency. An adaptive weighting strategy, inspired by curriculum learning, dynamically balances contrastive and imitation objectives for stable and effective training. Extensive experiments on simulated benchmarks (LIBERO and MimicGen) and real-world demonstrations validate that LaDA achieves strong performance and generalizes effectively to unseen or related tasks.