Enabling Deep Learning on Edge Devices through Filter Pruning and Knowledge Transfer
This addresses the need for personalized, responsive, and private training of state-of-the-art models on edge devices, representing an incremental improvement through hybrid methods.
The paper tackles the problem of enabling deep learning on resource-constrained edge devices by proposing a filter-pruning-based model compression method that removes up to 99.36% parameters while preserving over 90% Top-1 accuracy on CIFAR-10, and a knowledge transfer method that allows compressed models to converge in three to six minutes for incremental learning and classify unseen categories with 78.92% accuracy.
Deep learning models have introduced various intelligent applications to edge devices, such as image classification, speech recognition, and augmented reality. There is an increasing need of training such models on the devices in order to deliver personalized, responsive, and private learning. To address this need, this paper presents a new solution for deploying and training state-of-the-art models on the resource-constrained devices. First, the paper proposes a novel filter-pruning-based model compression method to create lightweight trainable models from large models trained in the cloud, without much loss of accuracy. Second, it proposes a novel knowledge transfer method to enable the on-device model to update incrementally in real time or near real time using incremental learning on new data and enable the on-device model to learn the unseen categories with the help of the in-cloud model in an unsupervised fashion. The results show that 1) our model compression method can remove up to 99.36% parameters of WRN-28-10, while preserving a Top-1 accuracy of over 90% on CIFAR-10; 2) our knowledge transfer method enables the compressed models to achieve more than 90% accuracy on CIFAR-10 and retain good accuracy on old categories; 3) it allows the compressed models to converge within real time (three to six minutes) on the edge for incremental learning tasks; 4) it enables the model to classify unseen categories of data (78.92% Top-1 accuracy) that it is never trained with.