Bridging Scale Discrepancies in Robotic Control via Language-Based Action Representations
This work addresses a domain-specific challenge in robotic manipulation by enabling more efficient pretraining and transfer across diverse robotic systems, though it is incremental as it builds on existing language model architectures.
The paper tackles the problem of distribution shifts in robotic action data due to numerical scale variations across platforms and tasks, proposing a language-based action representation that disregards numeric scales to improve generalization, resulting in significant performance gains in multi-task benchmarks.
Recent end-to-end robotic manipulation research increasingly adopts architectures inspired by large language models to enable robust manipulation. However, a critical challenge arises from severe distribution shifts between robotic action data, primarily due to substantial numerical variations in action commands across diverse robotic platforms and tasks, hindering the effective transfer of pretrained knowledge. To address this limitation, we propose a semantically grounded linguistic representation to normalize actions for efficient pretraining. Unlike conventional discretized action representations that are sensitive to numerical scales, the motion representation specifically disregards numeric scale effects, emphasizing directionality instead. This abstraction mitigates distribution shifts, yielding a more generalizable pretraining representation. Moreover, using the motion representation narrows the feature distance between action tokens and standard vocabulary tokens, mitigating modality gaps. Multi-task experiments on two benchmarks demonstrate that the proposed method significantly improves generalization performance and transferability in robotic manipulation tasks.