Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformers
This work addresses the high computational costs of DiTs for AI researchers and practitioners, offering an incremental improvement in inference efficiency without retraining.
The paper tackles the problem of computational inefficiency in Diffusion Transformers (DiTs) for image and video generation by proposing FoCa, a feature caching method that models hidden features as an ODE to improve stability under high acceleration ratios, achieving near-lossless speedups of up to 6.45 times on various models.
Diffusion Transformers (DiTs) have demonstrated exceptional performance in high-fidelity image and video generation. To reduce their substantial computational costs, feature caching techniques have been proposed to accelerate inference by reusing hidden representations from previous timesteps. However, current methods often struggle to maintain generation quality at high acceleration ratios, where prediction errors increase sharply due to the inherent instability of long-step forecasting. In this work, we adopt an ordinary differential equation (ODE) perspective on the hidden-feature sequence, modeling layer representations along the trajectory as a feature-ODE. We attribute the degradation of existing caching strategies to their inability to robustly integrate historical features under large skipping intervals. To address this, we propose FoCa (Forecast-then-Calibrate), which treats feature caching as a feature-ODE solving problem. Extensive experiments on image synthesis, video generation, and super-resolution tasks demonstrate the effectiveness of FoCa, especially under aggressive acceleration. Without additional training, FoCa achieves near-lossless speedups of 5.50 times on FLUX, 6.45 times on HunyuanVideo, 3.17 times on Inf-DiT, and maintains high quality with a 4.53 times speedup on DiT.