Learning representations by forward-propagating errors
This addresses the problem of inefficient neural network training on CPUs for researchers and practitioners, offering a potential alternative to GPU reliance.
The paper tackles the high computational cost and slow training speed of back-propagation on CPUs by proposing a new learning algorithm based on forward-propagating errors using dual numbers from algebraic geometry, achieving speed comparable to CUDA-accelerated GPU training.
Back-propagation (BP) is widely used learning algorithm for neural network optimization. However, BP requires enormous computation cost and is too slow to train in central processing unit (CPU). Therefore current neural network optimizaiton is performed in graphical processing unit (GPU) with compute unified device architecture (CUDA) programming. In this paper, we propose a light, fast learning algorithm on CPU that is fast as CUDA acceleration on GPU. This algorithm is based on forward-propagating method, using concept of dual number in algebraic geometry.