Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head Videos
This addresses the need for safe adoption of avatar generators in applications like AR/VR and videoconferencing by providing a verification mechanism to prevent unauthorized use of facial appearances.
The paper tackles the problem of verifying whether synthetic talking-head videos use a person's appearance without consent by introducing avatar fingerprinting, achieving an average AUC of 0.85 for identity verification and generalizing to unseen generators with an AUC of 0.83.
Modern avatar generators allow anyone to synthesize photorealistic real-time talking avatars, ushering in a new era of avatar-based human communication, such as with immersive AR/VR interactions or videoconferencing with limited bandwidths. Their safe adoption, however, requires a mechanism to verify if the rendered avatar is trustworthy: does it use the appearance of an individual without their consent? We term this task avatar fingerprinting. To tackle it, we first introduce a large-scale dataset of real and synthetic videos of people interacting on a video call, where the synthetic videos are generated using the facial appearance of one person and the expressions of another. We verify the identity driving the expressions in a synthetic video, by learning motion signatures that are independent of the facial appearance shown. Our solution, the first in this space, achieves an average AUC of 0.85. Critical to its practical use, it also generalizes to new generators never seen in training (average AUC of 0.83). The proposed dataset and other resources can be found at: https://research.nvidia.com/labs/nxp/avatar-fingerprinting/.