FUTURE-VLA: Forecasting Unified Trajectories Under Real-time Execution
This work is significant for robotics researchers and practitioners, as it enables real-time, long-horizon control and future forecasting for robotic systems, which was previously limited by computational latency.
This paper addresses the challenge of deploying general vision-language models for long-horizon control and future forecasting on robots, which is hindered by high latency. The authors introduce FUTURE-VLA, a unified architecture that achieves real-time predictive capabilities by using temporally adaptive compression and latent-space autoregression, resulting in state-of-the-art success rates of 99.2% on LIBERO, 75.4% on RoboTwin, and 78.0% on a real-world Piper platform, while extending the spatiotemporal window by 16x and maintaining single-frame inference latency.
General vision-language models increasingly support unified spatiotemporal reasoning over long video streams, yet deploying such capabilities on robots remains constrained by the prohibitive latency of processing long-horizon histories and generating high-dimensional future predictions. To bridge this gap, we present FUTURE-VLA, a unified architecture that reformulates long-horizon control and future forecasting as a monolithic sequence-generation task. Adopting a dual-sided efficiency paradigm, FUTURE-VLA leverages a temporally adaptive compression strategy to maximize spatiotemporal information density, enabling the ingestion of extensive multi-view histories while maintaining constant inference latency. Simultaneously, it performs latent-space autoregression to align actionable dynamics with reviewable visual look-aheads in a single forward pass. These real-time predictive capabilities further enable a prediction-guided Human-In-the-Loop mechanism via interactive execution gating, allowing operators to dynamically validate behaviors based on interpretable future previews. Extensive evaluations demonstrate that FUTURE-VLA establishes new state-of-the-art performance, attaining success rates of 99.2% on LIBERO, 75.4% on RoboTwin, and 78.0% on a real-world Piper platform, all with a $16\times$ extended spatiotemporal window while maintaining the inference latency of a single-frame baseline.