The eyes know it: FakeET -- An Eye-tracking Database to Understand Deepfake Perception
This work addresses the challenge of deepfake detection for security and media integrity by providing a novel dataset, though it is incremental as it builds on existing deepfake datasets.
The researchers tackled the problem of understanding human perception of deepfake videos by creating the FakeET database, which includes eye-tracking and EEG data from 40 users on 811 videos, revealing distinct patterns for real vs. fake videos and enabling forgery localization and detection.
We present \textbf{FakeET}-- an eye-tracking database to understand human visual perception of \emph{deepfake} videos. Given that the principal purpose of deepfakes is to deceive human observers, FakeET is designed to understand and evaluate the ease with which viewers can detect synthetic video artifacts. FakeET contains viewing patterns compiled from 40 users via the \emph{Tobii} desktop eye-tracker for 811 videos from the \textit{Google Deepfake} dataset, with a minimum of two viewings per video. Additionally, EEG responses acquired via the \emph{Emotiv} sensor are also available. The compiled data confirms (a) distinct eye movement characteristics for \emph{real} vs \emph{fake} videos; (b) utility of the eye-track saliency maps for spatial forgery localization and detection, and (c) Error Related Negativity (ERN) triggers in the EEG responses, and the ability of the \emph{raw} EEG signal to distinguish between \emph{real} and \emph{fake} videos.