A Survey of Early Exit Deep Neural Networks in NLP
It addresses the need for efficient models in resource-constrained NLP applications, but as a survey, it is incremental in summarizing existing methods.
This paper surveys early exit strategies in deep neural networks for NLP, which tackle the problem of high computational requirements by enabling adaptive inference to accelerate processing for simpler samples, thereby reducing latency and improving robustness.
Deep Neural Networks (DNNs) have grown increasingly large in size to achieve state of the art performance across a wide range of tasks. However, their high computational requirements make them less suitable for resource-constrained applications. Also, real-world datasets often consist of a mixture of easy and complex samples, necessitating adaptive inference mechanisms that account for sample difficulty. Early exit strategies offer a promising solution by enabling adaptive inference, where simpler samples are classified using the initial layers of the DNN, thereby accelerating the overall inference process. By attaching classifiers at different layers, early exit methods not only reduce inference latency but also improve the model robustness against adversarial attacks. This paper presents a comprehensive survey of early exit methods and their applications in NLP.