GazeSCRNN: Event-based Near-eye Gaze Tracking using a Spiking Neural Network
This work addresses gaze tracking for event-based systems, offering a novel approach with potential energy efficiency, but it is incremental as it builds on existing SNN and DVS methods.
The paper tackles near-eye gaze tracking by introducing GazeSCRNN, a spiking neural network that processes event streams from DVS cameras, achieving a Mean Angle Error of 6.034° and Mean Pupil Error of 2.094 mm on the EV-Eye dataset.
This work introduces GazeSCRNN, a novel spiking convolutional recurrent neural network designed for event-based near-eye gaze tracking. Leveraging the high temporal resolution, energy efficiency, and compatibility of Dynamic Vision Sensor (DVS) cameras with event-based systems, GazeSCRNN uses a spiking neural network (SNN) to address the limitations of traditional gaze-tracking systems in capturing dynamic movements. The proposed model processes event streams from DVS cameras using Adaptive Leaky-Integrate-and-Fire (ALIF) neurons and a hybrid architecture optimized for spatio-temporal data. Extensive evaluations on the EV-Eye dataset demonstrate the model's accuracy in predicting gaze vectors. In addition, we conducted ablation studies to reveal the importance of the ALIF neurons, dynamic event framing, and training techniques, such as Forward-Propagation-Through-Time, in enhancing overall system performance. The most accurate model achieved a Mean Angle Error (MAE) of 6.034° and a Mean Pupil Error (MPE) of 2.094 mm. Consequently, this work is pioneering in demonstrating the feasibility of using SNNs for event-based gaze tracking, while shedding light on critical challenges and opportunities for further improvement.