Block Cascading: Training Free Acceleration of Block-Causal Video Models
This addresses the problem of slow inference speeds in video generation for users, offering a practical acceleration solution, though it is incremental as it builds on existing block-causal methods.
The paper tackled the speed-quality trade-off in block-causal video generation by introducing Block Cascading, a training-free parallelization method that accelerates models from 16 to 30 FPS for 1.3B models and from 4.5 to 12.5 FPS for 14B models without significant quality loss.
Block-causal video generation faces a stark speed-quality trade-off: small 1.3B models manage only 16 FPS while large 14B models crawl at 4.5 FPS, forcing users to choose between responsiveness and quality. Block Cascading significantly mitigates this trade-off through training-free parallelization. Our key insight: future video blocks do not need fully denoised current blocks to begin generation. By starting block generation with partially denoised context from predecessors, we transform sequential pipelines into parallel cascades where multiple blocks denoise simultaneously. With 5 GPUs exploiting temporal parallelism, we achieve ~2x acceleration across all model scales: 1.3B models accelerate from 16 to 30 FPS, 14B models from 4.5 to 12.5 FPS. Beyond inference speed, Block Cascading eliminates overhead from KV-recaching (of ~200ms) during context switches for interactive generation. Extensive evaluations validated against multiple block-causal pipelines demonstrate no significant loss in generation quality when switching from block-causal to Block Cascading pipelines for inference. Project Page: https://hmrishavbandy.github.io/block_cascading_page/