Deep Learning: Computational Aspects
This is an incremental review article summarizing existing computational aspects for researchers and practitioners in deep learning.
The paper reviews computational challenges in deep learning, focusing on the intensive training requirements and the role of efficient linear algebra libraries, SGD, and batch sampling for handling massive datasets.
In this article we review computational aspects of Deep Learning (DL). Deep learning uses network architectures consisting of hierarchical layers of latent variables to construct predictors for high-dimensional input-output models. Training a deep learning architecture is computationally intensive, and efficient linear algebra libraries is the key for training and inference. Stochastic gradient descent (SGD) optimization and batch sampling are used to learn from massive data sets.