Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers
This addresses the problem of expensive large-scale pretraining for diffusion models, offering an efficient solution for researchers and practitioners, though it is incremental as it builds on existing token dropping and fusion techniques.
The paper tackles the high training cost of Diffusion Transformers (DiTs) by proposing SPRINT, a method that enables aggressive token dropping (up to 75%) to reduce computation while preserving generative quality, achieving 9.8x training savings with comparable FID/FDD on ImageNet-1K 256x256.
Diffusion Transformers (DiTs) deliver state-of-the-art generative performance but their quadratic training cost with sequence length makes large-scale pretraining prohibitively expensive. Token dropping can reduce training cost, yet naïve strategies degrade representations, and existing methods are either parameter-heavy or fail at high drop ratios. We present SPRINT, Sparse--Dense Residual Fusion for Efficient Diffusion Transformers, a simple method that enables aggressive token dropping (up to 75%) while preserving quality. SPRINT leverages the complementary roles of shallow and deep layers: early layers process all tokens to capture local detail, deeper layers operate on a sparse subset to cut computation, and their outputs are fused through residual connections. Training follows a two-stage schedule: long masked pre-training for efficiency followed by short full-token fine-tuning to close the train--inference gap. On ImageNet-1K 256x256, SPRINT achieves 9.8x training savings with comparable FID/FDD, and at inference, its Path-Drop Guidance (PDG) nearly halves FLOPs while improving quality. These results establish SPRINT as a simple, effective, and general solution for efficient DiT training.