A Study in Dataset Distillation for Image Super-Resolution
This work addresses memory and compute efficiency for generative restoration models, but it is incremental as it applies an existing concept to a new domain.
The study tackled dataset distillation for image super-resolution, showing that a distilled dataset at 8.88% of the original size can train models with nearly the same reconstruction fidelity as those using full datasets.
Dataset distillation aims to compress large datasets into compact yet highly informative subsets that preserve the training behavior of the original data. While this concept has gained traction in classification, its potential for image Super-Resolution (SR) remains largely untapped. In this work, we conduct the first systematic study of dataset distillation for SR, evaluating both pixel- and latent-space formulations. We show that a distilled dataset, occupying only 8.88% of the original size, can train SR models that retain nearly the same reconstruction fidelity as those trained on full datasets. Furthermore, we analyze how initialization strategies and distillation objectives affect efficiency, convergence, and visual quality. Our findings highlight the feasibility of SR dataset distillation and establish foundational insights for memory- and compute-efficient generative restoration models.