SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning
This work addresses the challenge of accelerating robotic manipulation policies without additional demonstrations, which is incremental but practical for robotics applications.
The paper tackles the problem of robotic policy execution speed lagging behind hardware capabilities by proposing SpeedAug, an RL-based framework that efficiently adapts pre-trained policies for faster task execution. Empirical results show it significantly improves sample efficiency while maintaining high success rates on robotic manipulation benchmarks.
Recent advances in robotic policy learning have enabled complex manipulation in real-world environments, yet the execution speed of these policies often lags behind hardware capabilities due to the cost of collecting faster demonstrations. Existing works on policy acceleration reinterpret action sequence for unseen execution speed, thereby encountering distributional shifts from the original demonstrations. Reinforcement learning is a promising approach that adapts policies for faster execution without additional demonstration, but its unguided exploration is sample inefficient. We propose SpeedAug, an RL-based policy acceleration framework that efficiently adapts pre-trained policies for faster task execution. SpeedAug constructs behavior prior that encompasses diverse tempos of task execution by pre-training a policy on speed-augmented demonstrations. Empirical results on robotic manipulation benchmarks show that RL fine-tuning initialized from this tempo-enriched policy significantly improves the sample efficiency of existing RL and policy acceleration methods while maintaining high success rate.