Mono-Forward: Backpropagation-Free Algorithm for Efficient Neural Network Training Harnessing Local Errors
This addresses the need for more efficient and biologically plausible training algorithms for machine learning practitioners, though it builds on existing forward-forward frameworks.
The paper tackles the problem of high memory costs and lack of biological plausibility in neural network training by introducing the Mono-Forward algorithm, a backpropagation-free method that matches or surpasses backpropagation accuracy across benchmarks like MNIST and CIFAR-100 with reduced memory usage.
Backpropagation is the standard method for achieving state-of-the-art accuracy in neural network training, but it often imposes high memory costs and lacks biological plausibility. In this paper, we introduce the Mono-Forward algorithm, a purely local layerwise learning method inspired by Hinton's Forward-Forward framework. Unlike backpropagation, Mono-Forward optimizes each layer solely with locally available information, eliminating the reliance on global error signals. We evaluated Mono-Forward on multi-layer perceptrons and convolutional neural networks across multiple benchmarks, including MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100. The test results show that Mono-Forward consistently matches or surpasses the accuracy of backpropagation across all tasks, with significantly reduced and more even memory usage, better parallelizability, and a comparable convergence rate.