Qi Qu

Huminhao Zhu, Fangyikang Wang, Tianyu Ding et al.

Training with synthetic data is becoming increasingly inevitable as synthetic content proliferates across the web, driven by the remarkable performance of recent deep generative models. This reliance on synthetic data can also be intentional, as seen in Rectified Flow models, whose Reflow method iteratively uses self-generated data to straighten the flow and improve sampling efficiency. However, recent studies have shown that repeatedly training on self-generated samples can lead to model collapse (MC), where performance degrades over time. Despite this, most recent work on MC either focuses on empirical observations or analyzes regression problems and maximum likelihood objectives, leaving a rigorous theoretical analysis of reflow methods unexplored. In this paper, we aim to fill this gap by providing both theoretical analysis and practical solutions for addressing MC in diffusion/flow models. We begin by studying Denoising Autoencoders and prove performance degradation when DAEs are iteratively trained on their own outputs. To the best of our knowledge, we are the first to rigorously analyze model collapse in DAEs and, by extension, in diffusion models and Rectified Flow. Our analysis and experiments demonstrate that rectified flow also suffers from MC, leading to potential performance degradation in each reflow step. Additionally, we prove that incorporating real data can prevent MC during recursive DAE training, supporting the recent trend of using real data as an effective approach for mitigating MC. Building on these insights, we propose a novel Real-data Augmented Reflow and a series of improved variants, which seamlessly integrate real data into Reflow training by leveraging reverse flow. Empirical evaluations on standard image benchmarks confirm that RA Reflow effectively mitigates model collapse, preserving high-quality sample generation even with fewer sampling steps.

Siyi Chen, Yixuan Jia, Qing Qu et al.

Data assimilation (DA) integrates observations with a dynamical model to estimate states of PDE-governed systems. Model-driven methods (e.g., Kalman, particle) presuppose full knowledge of the true dynamics, which is not always satisfied in practice, while purely data-driven solvers learn a deterministic mapping between observations and states and therefore miss the intrinsic stochasticity of real processes. Recently, score-based diffusion models learn a global diffusion prior and provide a good modeling of the stochastic dynamics, showing new potential for DA. However, their all-at-once generation rather than step-by-step transition limits their performance when dealing with highly complex stochastic processes and lacks physical interpretability. To tackle these drawbacks, we introduce FlowDAS, a generative DA framework that uses stochastic interpolants to directly learn state transition dynamics and achieve step-by-step transition to better model the real dynamics. We also improve the framework by combining the observation, better suiting the DA settings. Directly learning the underlying dynamics from collected data removes restrictive dynamical assumptions, and conditioning on observations at each interpolation step yields stable, measurement-consistent forecasts. Experiments on Lorenz-63, Navier-Stokes super-resolution/sparse-observation scenarios, and large-scale weather forecasting -- where dynamics are partly or wholly unknown -- show that FlowDAS surpasses model-driven methods, neural operators, and score-based baselines in accuracy and physical plausibility.

Huijie Zhang, Aliaksandr Siarohin, Willi Menapace et al.

MeanFlow has recently emerged as a powerful framework for few-step generative modeling trained from scratch, but its success is not yet fully understood. In this work, we show that the MeanFlow objective naturally decomposes into two parts: trajectory flow matching and trajectory consistency. Through gradient analysis, we find that these terms are strongly negatively correlated, causing optimization conflict and slow convergence. Motivated by these insights, we introduce $α$-Flow, a broad family of objectives that unifies trajectory flow matching, Shortcut Model, and MeanFlow under one formulation. By adopting a curriculum strategy that smoothly anneals from trajectory flow matching to MeanFlow, $α$-Flow disentangles the conflicting objectives, and achieves better convergence. When trained from scratch on class-conditional ImageNet-1K 256x256 with vanilla DiT backbones, $α$-Flow consistently outperforms MeanFlow across scales and settings. Our largest $α$-Flow-XL/2+ model achieves new state-of-the-art results using vanilla DiT backbones, with FID scores of 2.58 (1-NFE) and 2.15 (2-NFE).

Lianghe Shi, Meng Wu, Huijie Zhang et al.

The widespread use of diffusion models has led to an abundance of AI-generated data, raising concerns about model collapse -- a phenomenon in which recursive iterations of training on synthetic data lead to performance degradation. Prior work primarily characterizes this collapse via variance shrinkage or distribution shift, but these perspectives miss practical manifestations of model collapse. This paper identifies a transition from generalization to memorization during model collapse in diffusion models, where models increasingly replicate training data instead of generating novel content during iterative training on synthetic samples. This transition is directly driven by the declining entropy of the synthetic training data produced in each training cycle, which serves as a clear indicator of model degradation. Motivated by this insight, we propose an entropy-based data selection strategy to mitigate the transition from generalization to memorization and alleviate model collapse. Empirical results show that our approach significantly enhances visual quality and diversity in recursive generation, effectively preventing collapse.