LightFF: Lightweight Inference for Forward-Forward Algorithm
This work addresses energy efficiency constraints in AI systems, particularly for wearable technologies, but it is incremental as it builds on the existing Forward-Forward algorithm.
The paper tackles the high energy consumption of deep neural networks by proposing LightFF, a lightweight inference scheme for networks trained with the Forward-Forward algorithm, demonstrating its relevance on MNIST, CIFAR, and real-world applications like epileptic seizure detection and cardiac arrhythmia classification.
The human brain performs tasks with an outstanding energy efficiency, i.e., with approximately 20 Watts. The state-of-the-art Artificial/Deep Neural Networks (ANN/DNN), on the other hand, have recently been shown to consume massive amounts of energy. The training of these ANNs/DNNs is done almost exclusively based on the back-propagation algorithm, which is known to be biologically implausible. This has led to a new generation of forward-only techniques, including the Forward-Forward algorithm. In this paper, we propose a lightweight inference scheme specifically designed for DNNs trained using the Forward-Forward algorithm. We have evaluated our proposed lightweight inference scheme in the case of the MNIST and CIFAR datasets, as well as two real-world applications, namely, epileptic seizure detection and cardiac arrhythmia classification using wearable technologies, where complexity overheads/energy consumption is a major constraint, and demonstrate its relevance. Our code is available at https://github.com/AminAminifar/LightFF.