Towards High-Order Mean Flow Generative Models: Feasibility, Expressivity, and Provably Efficient Criteria
This work provides a theoretical foundation for high-order flow matching models, addressing the need for richer dynamics with practical sampling efficiency in generative AI, though it is incremental as it builds on the existing MeanFlow framework.
The paper tackles the problem of extending generative modeling to high-order dynamics by introducing Second-Order MeanFlow, which incorporates average acceleration fields, and proves its feasibility, expressivity via circuit complexity in the TC^0 class, and efficient implementation with attention approximations in time n^{2+o(1)}.
Generative modelling has seen significant advances through simulation-free paradigms such as Flow Matching, and in particular, the MeanFlow framework, which replaces instantaneous velocity fields with average velocities to enable efficient single-step sampling. In this work, we introduce a theoretical study on Second-Order MeanFlow, a novel extension that incorporates average acceleration fields into the MeanFlow objective. We first establish the feasibility of our approach by proving that the average acceleration satisfies a generalized consistency condition analogous to first-order MeanFlow, thereby supporting stable, one-step sampling and tractable loss functions. We then characterize its expressivity via circuit complexity analysis, showing that under mild assumptions, the Second-Order MeanFlow sampling process can be implemented by uniform threshold circuits within the $\mathsf{TC}^0$ class. Finally, we derive provably efficient criteria for scalable implementation by leveraging fast approximate attention computations: we prove that attention operations within the Second-Order MeanFlow architecture can be approximated to within $1/\mathrm{poly}(n)$ error in time $n^{2+o(1)}$. Together, these results lay the theoretical foundation for high-order flow matching models that combine rich dynamics with practical sampling efficiency.