Why should we add early exits to neural networks?
This provides a solution for deploying neural networks in time-constrained scenarios like 5G and Fog computing, but it is incremental as it builds on existing early exit techniques.
The paper tackles the problem of inefficient inference in deep neural networks by introducing early exits, which allow predictions at intermediate layers, resulting in significant reductions in inference time and improved training stability.
Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including: (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them.