Model Already Knows the Best Noise: Bayesian Active Noise Selection via Attention in Video Diffusion Model
This work addresses a specific bottleneck in video diffusion models for AI researchers and practitioners, offering an incremental improvement over existing methods.
The paper tackles the problem of initial noise selection in video diffusion models, which significantly affects output quality and prompt alignment, by proposing ANSE, a model-aware framework that selects high-quality noise seeds using attention-based uncertainty, improving video quality and temporal coherence with only an 8-13% increase in inference time.
The choice of initial noise significantly affects the quality and prompt alignment of video diffusion models, where different noise seeds for the same prompt can lead to drastically different generations. While recent methods rely on externally designed priors such as frequency filters or inter-frame smoothing, they often overlook internal model signals that indicate which noise seeds are inherently preferable. To address this, we propose ANSE (Active Noise Selection for Generation), a model-aware framework that selects high-quality noise seeds by quantifying attention-based uncertainty. At its core is BANSA (Bayesian Active Noise Selection via Attention), an acquisition function that measures entropy disagreement across multiple stochastic attention samples to estimate model confidence and consistency. For efficient inference-time deployment, we introduce a Bernoulli-masked approximation of BANSA that enables score estimation using a single diffusion step and a subset of attention layers. Experiments on CogVideoX-2B and 5B demonstrate that ANSE improves video quality and temporal coherence with only an 8% and 13% increase in inference time, respectively, providing a principled and generalizable approach to noise selection in video diffusion. See our project page: https://anse-project.github.io/anse-project/