TEA-DNN: the Quest for Time-Energy-Accuracy Co-optimized Deep Neural Networks
This work addresses the need for efficient deep learning on resource-constrained embedded devices, offering a method to balance performance metrics, though it is incremental as it builds on existing NAS and hardware acceleration approaches.
The paper tackles the problem of optimizing convolutional neural networks for embedded platforms by developing TEA-DNN, a neural-architecture search algorithm that co-optimizes execution time, energy consumption, and accuracy, resulting in Pareto-optimal models validated on hardware like NVIDIA Jetson TX2 and Intel Movidius Neural Compute Stick.
Embedded deep learning platforms have witnessed two simultaneous improvements. First, the accuracy of convolutional neural networks (CNNs) has been significantly improved through the use of automated neural-architecture search (NAS) algorithms to determine CNN structure. Second, there has been increasing interest in developing hardware accelerators for CNNs that provide improved inference performance and energy consumption compared to GPUs. Such embedded deep learning platforms differ in the amount of compute resources and memory-access bandwidth, which would affect performance and energy consumption of CNNs. It is therefore critical to consider the available hardware resources in the network architecture search. To this end, we introduce TEA-DNN, a NAS algorithm targeting multi-objective optimization of execution time, energy consumption, and classification accuracy of CNN workloads on embedded architectures. TEA-DNN leverages energy and execution time measurements on embedded hardware when exploring the Pareto-optimal curves across accuracy, execution time, and energy consumption and does not require additional effort to model the underlying hardware. We apply TEA-DNN for image classification on actual embedded platforms (NVIDIA Jetson TX2 and Intel Movidius Neural Compute Stick). We highlight the Pareto-optimal operating points that emphasize the necessity to explicitly consider hardware characteristics in the search process. To the best of our knowledge, this is the most comprehensive study of Pareto-optimal models across a range of hardware platforms using actual measurements on hardware to obtain objective values.