Color-Oriented Redundancy Reduction in Dataset Distillation
This work addresses a specific bottleneck in dataset distillation for improving training efficiency in image-based machine learning tasks, representing an incremental advancement.
The paper tackled the problem of color redundancy in dataset distillation by proposing AutoPalette, a framework that reduces redundancy at image and dataset levels, resulting in superior performance compared to existing methods in various datasets and evaluation scenarios.
Dataset Distillation (DD) is designed to generate condensed representations of extensive image datasets, enhancing training efficiency. Despite recent advances, there remains considerable potential for improvement, particularly in addressing the notable redundancy within the color space of distilled images. In this paper, we propose AutoPalette, a framework that minimizes color redundancy at the individual image and overall dataset levels, respectively. At the image level, we employ a palette network, a specialized neural network, to dynamically allocate colors from a reduced color space to each pixel. The palette network identifies essential areas in synthetic images for model training and consequently assigns more unique colors to them. At the dataset level, we develop a color-guided initialization strategy to minimize redundancy among images. Representative images with the least replicated color patterns are selected based on the information gain. A comprehensive performance study involving various datasets and evaluation scenarios is conducted, demonstrating the superior performance of our proposed color-aware DD compared to existing DD methods. The code is available at \url{https://github.com/KeViNYuAn0314/AutoPalette}.