OSCAR: Orthogonal Stochastic Control for Alignment-Respecting Diversity in Flow Matching
This addresses the costly and inefficient need for repeated sampling to discover diverse modes in text-to-image generation, offering a practical solution for users and developers, though it is incremental as it builds on existing flow-matching methods.
The paper tackled the problem of limited diversity in flow-based text-to-image models by introducing a training-free, inference-time control mechanism that boosts variation without degrading quality, resulting in consistent improvements in diversity metrics like Vendi Score and Brisque across multiple settings under fixed sampling budgets.
Flow-based text-to-image models follow deterministic trajectories, forcing users to repeatedly sample to discover diverse modes, which is a costly and inefficient process. We present a training-free, inference-time control mechanism that makes the flow itself diversity-aware. Our method simultaneously encourages lateral spread among trajectories via a feature-space objective and reintroduces uncertainty through a time-scheduled stochastic perturbation. Crucially, this perturbation is projected to be orthogonal to the generation flow, a geometric constraint that allows it to boost variation without degrading image details or prompt fidelity. Our procedure requires no retraining or modification to the base sampler and is compatible with common flow-matching solvers. Theoretically, our method is shown to monotonically increase a volume surrogate while, due to its geometric constraints, approximately preserving the marginal distribution. This provides a principled explanation for why generation quality is robustly maintained. Empirically, across multiple text-to-image settings under fixed sampling budgets, our method consistently improves diversity metrics such as the Vendi Score and Brisque over strong baselines, while upholding image quality and alignment.