Training-Time Action Conditioning for Efficient Real-Time Chunking
This work addresses efficiency in real-time robot control for applications like manipulation tasks, but it is incremental as it builds on existing real-time chunking methods.
The paper tackled the computational overhead of inference-time inpainting in real-time chunking for robot control by proposing training-time action conditioning, which simulates inference delay during training to eliminate runtime overhead. In real-world experiments with a VLA on tasks like box building and espresso making, it maintained task performance and speed parity while being computationally cheaper.
Real-time chunking (RTC) enables vision-language-action models (VLAs) to generate smooth, reactive robot trajectories by asynchronously predicting action chunks and conditioning on previously committed actions via inference-time inpainting. However, this inpainting method introduces computational overhead that increases inference latency. In this work, we propose a simple alternative: simulating inference delay at training time and conditioning on action prefixes directly, eliminating any inference-time overhead. Our method requires no modifications to the model architecture or robot runtime, and can be implemented with only a few additional lines of code. In simulated experiments, we find that training-time RTC outperforms inference-time RTC at higher inference delays. In real-world experiments on box building and espresso making tasks with the $π_{0.6}$ VLA, we demonstrate that training-time RTC maintains both task performance and speed parity with inference-time RTC while being computationally cheaper. Our results suggest that training-time action conditioning is a practical drop-in replacement for inference-time inpainting in real-time robot control.