Is Seeing Believing? Evaluating Human Sensitivity to Synthetic Video
This addresses the problem of disinformation from deepfakes for the public and policymakers, but it is incremental as it builds on existing concerns without introducing new mitigation strategies.
The paper investigated how human perception of synthetic videos (deepfakes) is affected by visual and auditory distortions, finding that these distortions and deepfake artifacts can reduce credibility in cognitive assessments.
Advances in machine learning have enabled the creation of realistic synthetic videos known as deepfakes. As deepfakes proliferate, concerns about rapid spread of disinformation and manipulation of public perception are mounting. Despite the alarming implications, our understanding of how individuals perceive synthetic media remains limited, obstructing the development of effective mitigation strategies. This paper aims to narrow this gap by investigating human responses to visual and auditory distortions of videos and deepfake-generated visuals and narration. In two between-subjects experiments, we study whether audio-visual distortions affect cognitive processing, such as subjective credibility assessment and objective learning outcomes. A third study reveals that artifacts from deepfakes influence credibility. The three studies show that video distortions and deepfake artifacts can reduce credibility. Our research contributes to the ongoing exploration of the cognitive processes involved in the evaluation and perception of synthetic videos, and underscores the need for further theory development concerning deepfake exposure.