Kaiyu Song

Kaiyu Song, Hanjiang Lai

Flow diffusion models (FDMs) have recently shown potential in generation tasks due to the high generation quality. However, the current ordinary differential equation (ODE) solver for FDMs, e.g., the Euler solver, still suffers from slow generation since ODE solvers need many number function evaluations (NFE) to keep high-quality generation. In this paper, we propose a novel training-free flow-solver to reduce NFE while maintaining high-quality generation. The key insight for the flow-solver is to leverage the previous steps to reduce the NFE, where a cache is created to reuse these results from the previous steps. Specifically, the Taylor expansion is first used to approximate the ODE. To calculate the high-order derivatives of Taylor expansion, the flow-solver proposes to use the previous steps and a polynomial interpolation to approximate it, where the number of orders we could approximate equals the number of previous steps we cached. We also prove that the flow-solver has a more minor approximation error and faster generation speed. Experimental results on the CIFAR-10, CelebA-HQ, LSUN-Bedroom, LSUN-Church, ImageNet, and real text-to-image generation prove the efficiency of the flow-solver. Specifically, the flow-solver improves the FID-30K from 13.79 to 6.75, from 46.64 to 19.49 with $\text{NFE}=10$ on CIFAR-10 and LSUN-Church, respectively.

Kaiyu Song, Hanjiang Lai, Yan Pan et al.

Training-free conditional generation based on flow matching aims to leverage pre-trained unconditional flow matching models to perform conditional generation without retraining. Recently, a successful training-free conditional generation approach incorporates conditions via posterior sampling, which relies on the availability of a score function in the unconditional diffusion model. However, flow matching models do not possess an explicit score function, rendering such a strategy inapplicable. Approximate posterior sampling for flow matching has been explored, but it is limited to linear inverse problems. In this paper, we propose Flow Matching-based Posterior Sampling (FMPS) to expand its application scope. We introduce a correction term by steering the velocity field. This correction term can be reformulated to incorporate a surrogate score function, thereby bridging the gap between flow matching models and score-based posterior sampling. Hence, FMPS enables the posterior sampling to be adjusted within the flow matching framework. Further, we propose two practical implementations of the correction mechanism: one aimed at improving generation quality, and the other focused on computational efficiency. Experimental results on diverse conditional generation tasks demonstrate that our method achieves superior generation quality compared to existing state-of-the-art approaches, validating the effectiveness and generality of FMPS.

Kaiyu Song, Hanjiang Lai

Training-free conditional diffusion models have received great attention in conditional image generation tasks. However, they require a computationally expensive conditional score estimator to let the intermediate results of each step in the reverse process toward the condition, which causes slow conditional generation. In this paper, we propose a novel Fisher information-based conditional diffusion (FICD) model to generate high-quality samples according to the condition. In particular, we further explore the conditional term from the perspective of Fisher information, where we show Fisher information can act as a weight to measure the informativeness of the condition in each generation step. According to this new perspective, we can control and gain more information along the conditional direction in the generation space. Thus, we propose the upper bound of the Fisher information to reformulate the conditional term, which increases the information gain and decreases the time cost. Experimental results also demonstrate that the proposed FICD can offer up to 2x speed-ups under the same sampling steps as most baselines. Meanwhile, FICD can improve the generation quality in various tasks compared to the baselines with a low computation cost.