Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation
For practitioners of video generation, DFD provides a simple fix to improve diversity and visual quality in distilled few-step models, though it is an incremental improvement over existing DMD methods.
Data-Forcing Distillation (DFD) restores diversity and fidelity in few-step video generation models by correcting mode collapse and over-saturation artifacts from DMD-based distillation, achieving better video dynamics and appearance than the teacher model with only 100-300 finetuning steps.
Recent progress has shown promise in distilling multi-step video diffusion models into efficient few-step students. Among them, Distribution Matching Distillation (DMD) and its successor DMD2 achieved strong generation quality and fast convergence. However, due to the nature of the reverse Kullback--Leibler (KL) objective, these methods exhibit two persistent failure modes: a substantial drop in sample diversity, and visibly over-saturated outputs that deviate from real-video appearance. In this work, we propose Data-Forcing Distillation (DFD), a simple post-training framework that restores diversity and fidelity in DMD with only a single-line of code change. At its core is the teacher score discrepancy to guide the student toward the real-data distribution, pulling it to missing modes (mitigating mode collapse) and away from problematic modes absent in real data (avoiding over-saturation). We provide an in-depth theoretical analysis of our framework and validate our approach on text-to-video, image-to-video, and autoregressive video generation. With only 100--300 steps of finetuning, DFD effectively restores diversity and fidelity on both Wan2.1-1.3B and Cosmos-Predict2.5-2B model, resolving the over-saturation artifacts with significantly better video dynamics and appearance, and even outperforms the teacher model.