Generative Dataset Distillation using Min-Max Diffusion Model
This work addresses dataset distillation for machine learning practitioners, but it is incremental as it builds on existing diffusion models and optimization techniques.
The paper tackles generative dataset distillation by using a diffusion model with a min-max loss to synthesize images, achieving second place in the ECCV2024 Dataset Distillation Challenge.
In this paper, we address the problem of generative dataset distillation that utilizes generative models to synthesize images. The generator may produce any number of images under a preserved evaluation time. In this work, we leverage the popular diffusion model as the generator to compute a surrogate dataset, boosted by a min-max loss to control the dataset's diversity and representativeness during training. However, the diffusion model is time-consuming when generating images, as it requires an iterative generation process. We observe a critical trade-off between the number of image samples and the image quality controlled by the diffusion steps and propose Diffusion Step Reduction to achieve optimal performance. This paper details our comprehensive method and its performance. Our model achieved $2^{nd}$ place in the generative track of \href{https://www.dd-challenge.com/#/}{The First Dataset Distillation Challenge of ECCV2024}, demonstrating its superior performance.