A scalable convolutional neural network for task-specified scenarios via knowledge distillation
This work addresses resource constraints in front-end visual systems by reducing network redundancy for specific tasks, but it is incremental as it builds on existing knowledge distillation methods.
The paper tackles the problem of redundancy in convolutional neural networks for specific vision tasks by proposing a task-specified knowledge distillation algorithm to create simplified models with pre-set computation costs and minimized accuracy loss, demonstrating feasibility on MNIST and CIFAR10 datasets.
In this paper, we explore the redundancy in convolutional neural network, which scales with the complexity of vision tasks. Considering that many front-end visual systems are interested in only a limited range of visual targets, the removing of task-specified network redundancy can promote a wide range of potential applications. We propose a task-specified knowledge distillation algorithm to derive a simplified model with pre-set computation cost and minimized accuracy loss, which suits the resource constraint front-end systems well. Experiments on the MNIST and CIFAR10 datasets demonstrate the feasibility of the proposed approach as well as the existence of task-specified redundancy.