Seeking Physics in Diffusion Noise
This work addresses the problem of enhancing physical plausibility in video generation for AI and computer vision applications, representing an incremental improvement by leveraging existing model features.
The paper investigated whether video diffusion models encode signals predictive of physical plausibility and found that physically plausible and implausible videos are partially separable in mid-layer feature spaces of a pretrained Diffusion Transformer. They introduced progressive trajectory selection, an inference-time strategy that improves physical consistency and reduces inference cost, achieving comparable results to Best-of-K sampling with substantially fewer denoising steps.
Do video diffusion models encode signals predictive of physical plausibility? We probe intermediate denoising representations of a pretrained Diffusion Transformer (DiT) and find that physically plausible and implausible videos are partially separable in mid-layer feature space across noise levels. This separability cannot be fully attributed to visual quality or generator identity, suggesting recoverable physics-related cues in frozen DiT features. Leveraging this observation, we introduce progressive trajectory selection, an inference-time strategy that scores parallel denoising trajectories at a few intermediate checkpoints using a lightweight physics verifier trained on frozen features, and prunes low-scoring candidates early. Extensive experiments on PhyGenBench demonstrate that our method improves physical consistency while reducing inference cost, achieving comparable results to Best-of-K sampling with substantially fewer denoising steps.