Dynamic Neural Network Architectural and Topological Adaptation and Related Methods -- A Survey
It addresses the problem of resource constraints in DNN applications for researchers and practitioners, but it is incremental as it surveys existing methods.
This survey provides an overview and categorization of state-of-the-art techniques for reducing time and space complexities in deep neural network training and inference, with a focus on architectural adaptations.
Training and inference in deep neural networks (DNNs) has, due to a steady increase in architectural complexity and data set size, lead to the development of strategies for reducing time and space requirements of DNN training and inference, which is of particular importance in scenarios where training takes place in resource constrained computation environments or inference is part of a time critical application. In this survey, we aim to provide a general overview and categorization of state-of-the-art (SOTA) of techniques to reduced DNN training and inference time and space complexities with a particular focus on architectural adaptions.