Early-exit deep neural networks for distorted images: providing an efficient edge offloading
This work addresses efficient edge-cloud inference for distorted images, offering an incremental improvement in early-exit DNNs for specific distortion handling.
The paper tackles the problem of edge offloading for deep neural networks (DNNs) with distorted images, which degrade accuracy estimates and increase cloud offloading, by introducing expert side branches trained on specific distortion types to improve robustness and offloading decisions, validated in a realistic scenario with Amazon EC2 instances.
Edge offloading for deep neural networks (DNNs) can be adaptive to the input's complexity by using early-exit DNNs. These DNNs have side branches throughout their architecture, allowing the inference to end earlier in the edge. The branches estimate the accuracy for a given input. If this estimated accuracy reaches a threshold, the inference ends on the edge. Otherwise, the edge offloads the inference to the cloud to process the remaining DNN layers. However, DNNs for image classification deals with distorted images, which negatively impact the branches' estimated accuracy. Consequently, the edge offloads more inferences to the cloud. This work introduces expert side branches trained on a particular distortion type to improve robustness against image distortion. The edge detects the distortion type and selects appropriate expert branches to perform the inference. This approach increases the estimated accuracy on the edge, improving the offloading decisions. We validate our proposal in a realistic scenario, in which the edge offloads DNN inference to Amazon EC2 instances.