Designing and Training of Lightweight Neural Networks on Edge Devices using Early Halting in Knowledge Distillation
This addresses the problem of high computation and energy costs for IoT applications, but it is incremental as it builds on existing knowledge distillation methods.
The paper tackles the challenge of deploying deep neural networks on edge devices by proposing a knowledge distillation-based training procedure with early halting to design and train lightweight DNNs, achieving adequate accuracy while considering storage, processing speed, and time constraints.
Automated feature extraction capability and significant performance of Deep Neural Networks (DNN) make them suitable for Internet of Things (IoT) applications. However, deploying DNN on edge devices becomes prohibitive due to the colossal computation, energy, and storage requirements. This paper presents a novel approach for designing and training lightweight DNN using large-size DNN. The approach considers the available storage, processing speed, and maximum allowable processing time to execute the task on edge devices. We present a knowledge distillation based training procedure to train the lightweight DNN to achieve adequate accuracy. During the training of lightweight DNN, we introduce a novel early halting technique, which preserves network resources; thus, speedups the training procedure. Finally, we present the empirically and real-world evaluations to verify the effectiveness of the proposed approach under different constraints using various edge devices.