A Survey of Techniques All Classifiers Can Learn from Deep Networks: Models, Optimizations, and Regularization
This incremental work addresses the problem of improving various classification algorithms by integrating deep learning techniques, benefiting researchers and practitioners in machine learning.
The paper surveys how non-neural network classifiers can adopt tools from deep learning, such as feature learning, optimization, and regularization methods, to enhance their performance and generality, identifying opportunities and challenges for future research.
Deep neural networks have introduced novel and useful tools to the machine learning community. Other types of classifiers can potentially make use of these tools as well to improve their performance and generality. This paper reviews the current state of the art for deep learning classifier technologies that are being used outside of deep neural networks. Non-network classifiers can employ many components found in deep neural network architectures. In this paper, we review the feature learning, optimization, and regularization methods that form a core of deep network technologies. We then survey non-neural network learning algorithms that make innovative use of these methods to improve classification. Because many opportunities and challenges still exist, we discuss directions that can be pursued to expand the area of deep learning for a variety of classification algorithms.