DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillation
This work addresses the overlooked issue of robustness in dataset distillation for researchers and practitioners in machine learning, though it is incremental as it builds on existing benchmarks.
The paper tackles the problem of evaluating adversarial robustness in dataset distillation by introducing DD-RobustBench, a comprehensive benchmark that expands on prior efforts with more methods, attacks, and datasets, and discovers that adding distilled data to training can improve robustness.
Dataset distillation is an advanced technique aimed at compressing datasets into significantly smaller counterparts, while preserving formidable training performance. Significant efforts have been devoted to promote evaluation accuracy under limited compression ratio while overlooked the robustness of distilled dataset. In this work, we introduce a comprehensive benchmark that, to the best of our knowledge, is the most extensive to date for evaluating the adversarial robustness of distilled datasets in a unified way. Our benchmark significantly expands upon prior efforts by incorporating a wider range of dataset distillation methods, including the latest advancements such as TESLA and SRe2L, a diverse array of adversarial attack methods, and evaluations across a broader and more extensive collection of datasets such as ImageNet-1K. Moreover, we assessed the robustness of these distilled datasets against representative adversarial attack algorithms like PGD and AutoAttack, while exploring their resilience from a frequency perspective. We also discovered that incorporating distilled data into the training batches of the original dataset can yield to improvement of robustness.