Semantic Contextualization of Face Forgery: A New Definition, Dataset, and Detection Method
This work addresses the need for a clearer definition and better detection of face forgeries in digital media, which is crucial for security and authenticity applications, though it is incremental in building on existing detection frameworks.
The paper tackles the problem of defining and detecting face forgeries by introducing a semantic context-based definition, constructing a new dataset with hierarchical labels, and proposing a semantics-oriented detection method. The results show that the dataset exposes weaknesses in current detectors and improves their generalizability, while the proposed method outperforms traditional classification-based detectors.
In recent years, deep learning has greatly streamlined the process of manipulating photographic face images. Aware of the potential dangers, researchers have developed various tools to spot these counterfeits. Yet, none asks the fundamental question: What digital manipulations make a real photographic face image fake, while others do not? In this paper, we put face forgery in a semantic context and define that computational methods that alter semantic face attributes to exceed human discrimination thresholds are sources of face forgery. Following our definition, we construct a large face forgery image dataset, where each image is associated with a set of labels organized in a hierarchical graph. Our dataset enables two new testing protocols to probe the generalizability of face forgery detectors. Moreover, we propose a semantics-oriented face forgery detection method that captures label relations and prioritizes the primary task (i.e., real or fake face detection). We show that the proposed dataset successfully exposes the weaknesses of current detectors as the test set and consistently improves their generalizability as the training set. Additionally, we demonstrate the superiority of our semantics-oriented method over traditional binary and multi-class classification-based detectors.