Oleg Komogortsev

Kateryna Melnyk, Lee Friedman, Oleg Komogortsev

Gaze prediction is essential for addressing motion-to-photon latency and ensuring seamless foveated rendering in Virtual Reality. The reliability of gaze forecasting is highly sensitive to individual differences and the eye movements being predicted. We evaluate recurrent, transformer-based, and classification-guided architectures to assess their generalization capabilities across oculomotor events. Using the GazeBase VR and Meta Quest Pro datasets, we analyzed the relationship between the median (P50) and high-percentile (P95) error profiles across subjects. The analysis reveals significant performance variability, showing that subjects with low P50 errors do not always exhibit the lowest extreme-case errors. Consequently, low median errors do not guarantee the robustness of the utilized solution. We discuss inference performance and address the class imbalance problem in short-term gaze prediction. These results identify a gap in standardized evaluation methods, necessitating a shift toward P95-focused, subject-specific metrics to develop reliable and perceptually stable gaze-contingent systems.

Dillon Lohr, Michael J. Proulx, Oleg Komogortsev

This paper performs the crucial work of establishing a baseline for gaze-driven authentication performance to begin answering fundamental research questions using a very large dataset of gaze recordings from 9202 people with a level of eye tracking (ET) signal quality equivalent to modern consumer-facing virtual reality (VR) platforms. The size of the employed dataset is at least an order-of-magnitude larger than any other dataset from previous related work. Binocular estimates of the optical and visual axes of the eyes and a minimum duration for enrollment and verification are required for our model to achieve a false rejection rate (FRR) of below 3% at a false acceptance rate (FAR) of 1 in 50,000. In terms of identification accuracy which decreases with gallery size, we estimate that our model would fall below chance-level accuracy for gallery sizes of 148,000 or more. Our major findings indicate that gaze authentication can be as accurate as required by the FIDO standard when driven by a state-of-the-art machine learning architecture and a sufficiently large training dataset.

Kateryna Melnyk, Lee Friedman, Dmytro Katrychuk et al.

Eye movement prediction is a promising area of research with the potential to improve performance and the user experience of systems based on eye-tracking technology. In this study, we analyze individual differences in gaze prediction performance. We use three fundamentally different models within the analysis: the lightweight Long Short-Term Memory network (LSTM), the transformer-based network for multivariate time series representation learning (TST), and the Oculomotor Plant Mathematical Model wrapped in the Kalman Filter framework (OPKF). Each solution was assessed on different eye-movement types. We show important subject-to-subject variation for all models and eye-movement types. We found that fixation noise is associated with poorer gaze prediction in fixation. For saccades, higher velocities are associated with poorer gaze prediction performance. We think these individual differences are important and propose that future research should report statistics related to inter-subject variation. We also propose that future models should be designed to reduce subject-to-subject variation.

Samantha Aziz, Oleg Komogortsev

We present a privacy-enhancing mechanism for gaze signals using a latent-noise autoencoder that prevents users from being re-identified across play sessions without their consent, while retaining the usability of the data for benign tasks. We evaluate privacy-utility trade-offs across biometric identification and gaze prediction tasks, showing that our approach significantly reduces biometric identifiability with minimal utility degradation. Unlike prior methods in this direction, our framework retains physiologically plausible gaze patterns suitable for downstream use, which produces favorable privacy-utility trade-off. This work advances privacy in gaze-based systems by providing a usable and effective mechanism for protecting sensitive gaze data.

Henry Griffith, Dillon Lohr, Evgeny Abdulin et al.

This manuscript presents GazeBase, a large-scale longitudinal dataset containing 12,334 monocular eye-movement recordings captured from 322 college-aged subjects. Subjects completed a battery of seven tasks in two contiguous sessions during each round of recording, including a - 1) fixation task, 2) horizontal saccade task, 3) random oblique saccade task, 4) reading task, 5/6) free viewing of cinematic video task, and 7) gaze-driven gaming task. A total of nine rounds of recording were conducted over a 37 month period, with subjects in each subsequent round recruited exclusively from the prior round. All data was collected using an EyeLink 1000 eye tracker at a 1,000 Hz sampling rate, with a calibration and validation protocol performed before each task to ensure data quality. Due to its large number of subjects and longitudinal nature, GazeBase is well suited for exploring research hypotheses in eye movement biometrics, along with other emerging applications applying machine learning techniques to eye movement signal analysis.

Ye Zhu, Yan Yan, Oleg Komogortsev

In this work, we tackle the problem of ternary eye movement classification, which aims to separate fixations, saccades and smooth pursuits from the raw eye positional data. The efficient classification of these different types of eye movements helps to better analyze and utilize the eye tracking data. Different from the existing methods that detect eye movement by several pre-defined threshold values, we propose a hierarchical Hidden Markov Model (HMM) statistical algorithm for detecting fixations, saccades and smooth pursuits. The proposed algorithm leverages different features from the recorded raw eye tracking data with a hierarchical classification strategy, separating one type of eye movement each time. Experimental results demonstrate the effectiveness and robustness of the proposed method by achieving competitive or better performance compared to the state-of-the-art methods.

Alex Karpov, Jacob Liberman, Dillon Lohr et al.

The usage of eye tracking sensors is expected to grow in virtual (VR) and augmented reality (AR) platforms. Provided that users of these platforms consent to employing captured eye movement signals for authentication and health assessment, it becomes important to estimate oculomotor plant and brain function characteristics in real time. This paper shows a path toward that goal by presenting a parallel processing architecture capable of estimating oculomotor plant characteristics and comparing its performance to a single-threaded implementation. Results show that the parallel implementation improves the speed, accuracy, and throughput of oculomotor plant characteristic estimation versus the original serial version for both large-scale and real-time simulation.

Samuel-Hunter Berndt, Douglas Kirkpatrick, Timothy Taviano et al.

The proper classification of major eye movements, saccades, fixations, and smooth pursuits, remains essential to utilizing eye-tracking data. There is difficulty in separating out smooth pursuits from the other behavior types, particularly from fixations. To this end, we propose a new offline algorithm, I-VDT-HMM, for tertiary classification of eye movements. The algorithm combines the simplicity of two foundational algorithms, I-VT and I-DT, as has been implemented in I-VDT, with the statistical predictive power of the Viterbi algorithm. We evaluate the fitness across a dataset of eight eye movement records at eight sampling rates gathered from previous research, with a comparison to the current state-of-the-art using the proposed quantitative and qualitative behavioral scores. The proposed algorithm achieves promising results in clean high sampling frequency data and with slight modifications could show similar results with lower quality data. Though, the statistical aspect of the algorithm comes at a cost of classification time.

Evgeniy Abdulin, Lee Friedman, Oleg Komogortsev

We built a custom video-based eye-tracker that saves every video frame as a full resolution image (MJPEG). Images can be processed offline for the detection of ocular features, including the pupil and corneal reflection (First Purkinje Image, P1) position. A comparison of multiple algorithms for detection of pupil and corneal reflection can be performed. The system provides for highly flexible stimulus creation, with mixing of graphic, image, and video stimuli. We can change cameras and infrared illuminators depending on the image qualities and frame rate desired. Using this system, we have detected the position of the Fourth Purkinje image (P4) in the frames. We show that when we estimate gaze by calculating P1-P4, signal compares well with gaze estimated with a DPI eye-tracker, which natively detects and tracks the P1 and P4.

Lee Friedman, Oleg Komogortsev

We create synthetic biometric databases to study general, fundamental, biometric principles. First, we check the validity of the synthetic database design by comparing it to real data in terms of biometric performance. The real data used for this validity check was from an eye-movement related biometric database. Next, we employ our database to evaluate the impact of variations of temporal persistence of features on biometric performance. We index temporal persistence with the intraclass correlation coefficient (ICC). We find that variations in temporal persistence are extremely highly correlated with variations in biometric performance. Finally, we use our synthetic database strategy to determine how many features are required to achieve particular levels of performance as the number of subjects in the database increases from 100 to 10,000. An important finding is that the number of features required to achieve various EER values (2%, 0.3%, 0.15%) is essentially constant in the database sizes that we studied. We hypothesize that the insights obtained from our study would be applicable to many biometric modalities where extracted feature properties resemble the properties of the synthetic features we discuss in this work.

Ioannis Rigas, Lee Friedman, Oleg Komogortsev

This work presents a study on the extraction and analysis of a set of 101 categories of eye movement features from three types of eye movement events: fixations, saccades, and post-saccadic oscillations. The eye movements were recorded during a reading task. For the categories of features with multiple instances in a recording we extract corresponding feature subtypes by calculating descriptive statistics on the distributions of these instances. A unified framework of detailed descriptions and mathematical formulas are provided for the extraction of the feature set. The analysis of feature values is performed using a large database of eye movement recordings from a normative population of 298 subjects. We demonstrate the central tendency and overall variability of feature values over the experimental population, and more importantly, we quantify the test-retest reliability (repeatability) of each separate feature. The described methods and analysis can provide valuable tools in fields exploring the eye movements, such as in behavioral studies, attention and cognition research, medical research, biometric recognition, and human-computer interaction.