Learning Using a Single Forward Pass
This addresses the need for efficient learning algorithms in resource-constrained edge AI applications, though it appears incremental as it builds on existing local learning methods.
The paper tackles the problem of resource-intensive backpropagation in deep learning by proposing SPELA, a single-forward-pass algorithm that uses local loss functions to update weights, achieving equivalent performance to backpropagation on datasets like CIFAR-10 and SVHN while using less memory.
We propose a learning algorithm to overcome the limitations of traditional backpropagation in resource-constrained environments: Solo Pass Embedded Learning Algorithm (SPELA). SPELA operates with local loss functions to update weights, significantly saving on resources allocated to the propagation of gradients and storing computational graphs while being sufficiently accurate. Consequently, SPELA can closely match backpropagation using less memory. Moreover, SPELA can effectively fine-tune pre-trained image recognition models for new tasks. Further, SPELA is extended with significant modifications to train CNN networks, which we evaluate on CIFAR-10, CIFAR-100, and SVHN 10 datasets, showing equivalent performance compared to backpropagation. Our results indicate that SPELA, with its features such as local learning and early exit, is a potential candidate for learning in resource-constrained edge AI applications.