Eye-gaze Estimation with HEOG and Neck EMG using Deep Neural Networks
This work addresses the challenge of enabling eye-gaze steering hearing aids for hearing-impaired individuals, representing an incremental improvement over existing methods by integrating new sensor data and deep learning.
The paper tackled the problem of unreliable eye-gaze estimation for hearing-impaired listeners in multi-talker scenes by using deep neural networks to combine horizontal electrooculography (HEOG) and neck electromyography (NEMG) measurements, achieving accuracies of up to 93.3% with HEOG and inertial measurement unit and 72.6% with HEOG and NEMG together.
Hearing-impaired listeners usually have troubles attending target talker in multi-talker scenes, even with hearing aids (HAs). The problem can be solved with eye-gaze steering HAs, which requires listeners eye-gazing on the target. In a situation where head rotates, eye-gaze is subject to both behaviors of saccade and head rotation. However, existing methods of eye-gaze estimation did not work reliably, since the listener's strategy of eye-gaze varies and measurements of the two behaviors were not properly combined. Besides, existing methods were based on hand-craft features, which could overlook some important information. In this paper, a head-fixed and a head-free experiments were conducted. We used horizontal electrooculography (HEOG) and neck electromyography (NEMG), which separately measured saccade and head rotation to commonly estimate eye-gaze. Besides traditional classifier and hand-craft features, deep neural networks (DNN) were introduced to automatically extract features from intact waveforms. Evaluation results showed that when the input was HEOG with inertial measurement unit, the best performance of our proposed DNN classifiers achieved 93.3%; and when HEOG was with NEMG together, the accuracy reached 72.6%, higher than that with HEOG (about 71.0%) or NEMG (about 35.7%) alone. These results indicated the feasibility to estimate eye-gaze with HEOG and NEMG.