Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control
This enables fast and accurate online processing of sEMG signals for smooth prosthetic hand control, representing an incremental advance in integrating Transformers with spiking neural networks for energy-efficient temporal signal processing.
The paper tackled the problem of making Transformers suitable for online signal processing by replacing self-attention with a sliding window attention mechanism, achieving state-of-the-art accuracy on a finger position regression dataset with sEMG signals while using only 3.5 ms time windows and reducing synaptic operations by up to 5.3 times without accuracy loss.
Transformers are state-of-the-art networks for most sequence processing tasks. However, the self-attention mechanism often used in Transformers requires large time windows for each computation step and thus makes them less suitable for online signal processing compared to Recurrent Neural Networks (RNNs). In this paper, instead of the self-attention mechanism, we use a sliding window attention mechanism. We show that this mechanism is more efficient for continuous signals with finite-range dependencies between input and target, and that we can use it to process sequences element-by-element, this making it compatible with online processing. We test our model on a finger position regression dataset (NinaproDB8) with Surface Electromyographic (sEMG) signals measured on the forearm skin to estimate muscle activities. Our approach sets the new state-of-the-art in terms of accuracy on this dataset while requiring only very short time windows of 3.5 ms at each inference step. Moreover, we increase the sparsity of the network using Leaky-Integrate and Fire (LIF) units, a bio-inspired neuron model that activates sparsely in time solely when crossing a threshold. We thus reduce the number of synaptic operations up to a factor of $\times5.3$ without loss of accuracy. Our results hold great promises for accurate and fast online processing of sEMG signals for smooth prosthetic hand control and is a step towards Transformers and Spiking Neural Networks (SNNs) co-integration for energy efficient temporal signal processing.