DLL: A Blazing Fast Deep Neural Network Library
This work addresses the need for faster deep learning tools for researchers and practitioners, but it is incremental as it focuses on software optimizations rather than algorithmic breakthroughs.
The authors tackled the problem of slow runtime in deep neural network training and inference by developing DLL, a library that uses novel software engineering strategies, resulting in systematic and significant speed improvements on CPU and GPU across three datasets and four models compared to five popular frameworks, with similar classification accuracies.
Deep Learning Library (DLL) is a new library for machine learning with deep neural networks that focuses on speed. It supports feed-forward neural networks such as fully-connected Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs). It also has very comprehensive support for Restricted Boltzmann Machines (RBMs) and Convolutional RBMs. Our main motivation for this work was to propose and evaluate novel software engineering strategies with potential to accelerate runtime for training and inference. Such strategies are mostly independent of the underlying deep learning algorithms. On three different datasets and for four different neural network models, we compared DLL to five popular deep learning frameworks. Experimentally, it is shown that the proposed framework is systematically and significantly faster on CPU and GPU. In terms of classification performance, similar accuracies as the other frameworks are reported.