Quantization-Free Autoregressive Action Transformer
This work addresses a bottleneck in imitation learning for robotics by simplifying the pipeline and improving performance, though it appears incremental as it builds on existing transformer and GIVT methods.
The paper tackles the problem of discrete action representations limiting generative models in transformer-based imitation learning by proposing a quantization-free method using Generative Infinite-Vocabulary Transformers (GIVT) as a continuous policy parametrization, achieving state-of-the-art performance on simulated robotics tasks.
Current transformer-based imitation learning approaches introduce discrete action representations and train an autoregressive transformer decoder on the resulting latent code. However, the initial quantization breaks the continuous structure of the action space thereby limiting the capabilities of the generative model. We propose a quantization-free method instead that leverages Generative Infinite-Vocabulary Transformers (GIVT) as a direct, continuous policy parametrization for autoregressive transformers. This simplifies the imitation learning pipeline while achieving state-of-the-art performance on a variety of popular simulated robotics tasks. We enhance our policy roll-outs by carefully studying sampling algorithms, further improving the results.