FastSTAR: Spatiotemporal Token Pruning for Efficient Autoregressive Video Synthesis
This work provides an incremental improvement in efficiency for STAR-based video synthesis, benefiting researchers and practitioners working on high-resolution video generation.
The paper addresses the "token explosion" problem in Spacetime Autoregressive modeling (STAR) for video generation, which causes a computational bottleneck. They propose FastSTAR, a training-free acceleration framework that uses Spatiotemporal Token Pruning to identify essential tokens, achieving up to a 2.01x speedup with less than 1% performance degradation on InfinityStar.
Visual Autoregressive modeling (VAR) has emerged as a highly efficient alternative to diffusion-based frameworks, achieving comparable synthesis quality. However, as this paradigm extends to Spacetime Autoregressive modeling (STAR) for video generation, scaling resolution and frame counts leads to a "token explosion" that creates a massive computational bottleneck in the final refinement stages. To address this, we propose FastSTAR, a training-free acceleration framework designed for high-quality video generation. Our core method, Spatiotemporal Token Pruning, identifies essential tokens by integrating two specialized terms: (1) Spatial similarity, which evaluates structural convergence across hierarchical scales to skip computations in regions where further refinement becomes redundant, and (2) Temporal similarity, which identifies active motion trajectories by assessing feature-level variations relative to the preceding clip. Combined with a Partial Update mechanism, FastSTAR ensures that only non-converged regions are refined, maintaining fluid motion while bypassing redundant computations. Experimental results on InfinityStar demonstrate that FastSTAR achieves up to a 2.01x speedup with a PSNR of 28.29 and less than 1% performance degradation, proving a superior efficiency-quality trade-off for STAR-based video synthesis.