Dream Distillation: A Data-Independent Model Compression Framework
This addresses the problem of deploying deep learning on IoT-devices when data is unavailable, offering a novel solution for privacy-sensitive applications.
The paper tackles model compression without access to real data, such as in private data scenarios, by proposing Dream Distillation, achieving 88.5% accuracy on CIFAR-10 without training on the original data.
Model compression is eminently suited for deploying deep learning on IoT-devices. However, existing model compression techniques rely on access to the original or some alternate dataset. In this paper, we address the model compression problem when no real data is available, e.g., when data is private. To this end, we propose Dream Distillation, a data-independent model compression framework. Our experiments show that Dream Distillation can achieve 88.5% accuracy on the CIFAR-10 test set without actually training on the original data!