Bezier Distillation
This work aims to improve performance in flow models for accelerating sampling, but it appears incremental as it builds on existing distillation and curve methods.
The paper tackles the problem of error accumulation in Rectified Flow when distilling mapping relationships into neural networks, proposing to combine multi-teacher knowledge distillation with Bezier curves to address this issue, though no concrete results or numbers are provided as the work is still in progress.
In Rectified Flow, by obtaining the rectified flow several times, the mapping relationship between distributions can be distilled into a neural network, and the target distribution can be directly predicted by the straight lines of the flow. However, during the pairing process of the mapping relationship, a large amount of error accumulation will occur, resulting in a decrease in performance after multiple rectifications. In the field of flow models, knowledge distillation of multi - teacher diffusion models is also a problem worthy of discussion in accelerating sampling. I intend to combine multi - teacher knowledge distillation with Bezier curves to solve the problem of error accumulation. Currently, the related paper is being written by myself.