Reshaping the Forward-Forward Algorithm with a Similarity-Based Objective
This work addresses biologically plausible alternatives to backpropagation for neural networks, though it is incremental as it builds on the existing Forward-Forward algorithm.
The paper tackled the accuracy and efficiency limitations of the Forward-Forward algorithm by integrating it with similarity learning, eliminating multiple forward passes during inference. The proposed FAUST method achieved 56.22% accuracy on CIFAR-10, narrowing the gap with backpropagation's 57.63%.
Backpropagation is the pivotal algorithm underpinning the success of artificial neural networks, yet it has critical limitations such as biologically implausible backward locking and global error propagation. To circumvent these constraints, the Forward-Forward algorithm was proposed as a more biologically plausible method that replaces the backward pass with an additional forward pass. Despite this advantage, the Forward-Forward algorithm significantly trails backpropagation in accuracy, and its optimal form exhibits low inference efficiency due to multiple forward passes required. In this work, the Forward-Forward algorithm is reshaped through its integration with similarity learning frameworks, eliminating the need for multiple forward passes during inference. This proposed algorithm is named Forward-Forward Algorithm Unified with Similarity-based Tuplet loss (FAUST). Empirical evaluations on MNIST, Fashion-MNIST, and CIFAR-10 datasets indicate that FAUST substantially improves accuracy, narrowing the gap with backpropagation. On CIFAR-10, FAUST achieves 56.22\% accuracy with a simple multi-layer perceptron architecture, approaching the backpropagation benchmark of 57.63\% accuracy.