FVD: Inference-Time Alignment of Diffusion Models via Fleming-Viot Resampling
This work addresses a specific bottleneck in diffusion model alignment for researchers and practitioners, offering a more efficient and effective method for tasks like image generation, though it is incremental in nature.
The paper tackles the problem of diversity collapse in Sequential Monte Carlo-based diffusion samplers by introducing Fleming-Viot Diffusion (FVD), an inference-time alignment method that replaces multinomial resampling with a birth-death mechanism, resulting in improved performance such as a 7% gain in ImageReward on DrawBench and up to 66 times faster inference than value-based approaches.
We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte Carlo (SMC) based diffusion samplers. Existing SMC-based diffusion samplers often rely on multinomial resampling or closely related resampling schemes, which can still reduce diversity and lead to lineage collapse under strong selection pressure. Inspired by Fleming-Viot population dynamics, FVD replaces multinomial resampling with a specialized birth-death mechanism designed for diffusion alignment. To handle cases where rewards are only approximately available and naive rebirth would collapse deterministic trajectories, FVD integrates independent reward-based survival decisions with stochastic rebirth noise. This yields flexible population dynamics that preserve broader trajectory support while effectively exploring reward-tilted distributions, all without requiring value function approximation or costly rollouts. FVD is fully parallelizable and scales efficiently with inference compute. Empirically, it achieves substantial gains across settings: on DrawBench it outperforms prior methods by 7% in ImageReward, while on class-conditional tasks it improves FID by roughly 14-20% over strong baselines and is up to 66 times faster than value-based approaches.