DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference
This addresses energy and latency inefficiencies in edge-cloud collaborative inference for DNN models, offering significant improvements but is incremental as it builds on existing DVFS and offloading techniques.
The paper tackles the challenge of optimizing DNN inference for energy consumption and latency on edge devices by proposing DVFO, a DVFS-enabled edge-cloud collaborative framework that uses deep reinforcement learning to co-optimize device frequencies and offloading parameters, resulting in an average 33% energy reduction and up to 59.1% latency reduction while maintaining accuracy.
Due to limited resources on edge and different characteristics of deep neural network (DNN) models, it is a big challenge to optimize DNN inference performance in terms of energy consumption and end-to-end latency on edge devices. In addition to the dynamic voltage frequency scaling (DVFS) technique, the edge-cloud architecture provides a collaborative approach for efficient DNN inference. However, current edge-cloud collaborative inference methods have not optimized various compute resources on edge devices. Thus, we propose DVFO, a novel DVFS-enabled edge-cloud collaborative inference framework, which co-optimizes DVFS and offloading parameters via deep reinforcement learning (DRL). Specifically, DVFO automatically co-optimizes 1) the CPU, GPU and memory frequencies of edge devices, and 2) the feature maps to be offloaded to cloud servers. In addition, it leverages a thinking-while-moving concurrent mechanism to accelerate the DRL learning process, and a spatial-channel attention mechanism to extract DNN feature maps of secondary importance for workload offloading. This approach improves inference performance for different DNN models under various edge-cloud network conditions. Extensive evaluations using two datasets and six widely-deployed DNN models on three heterogeneous edge devices show that DVFO significantly reduces the energy consumption by 33% on average, compared to state-of-the-art schemes. Moreover, DVFO achieves up to 28.6%-59.1% end-to-end latency reduction, while maintaining accuracy within 1% loss on average.