Characterizing the temporal dynamics of universal speech representations for generalizable deepfake detection
This addresses the generalizability issue in deepfake detection for security applications, but it is incremental as it builds on existing representation methods.
The paper tackled the problem of deepfake speech detection lacking generalizability to unseen generative attacks by characterizing the long-term temporal dynamics of universal speech representations, showing that different generative models produce similar dynamics patterns and improving detection on unseen methods in ASVspoof datasets.
Existing deepfake speech detection systems lack generalizability to unseen attacks (i.e., samples generated by generative algorithms not seen during training). Recent studies have explored the use of universal speech representations to tackle this issue and have obtained inspiring results. These works, however, have focused on innovating downstream classifiers while leaving the representation itself untouched. In this study, we argue that characterizing the long-term temporal dynamics of these representations is crucial for generalizability and propose a new method to assess representation dynamics. Indeed, we show that different generative models generate similar representation dynamics patterns with our proposed method. Experiments on the ASVspoof 2019 and 2021 datasets validate the benefits of the proposed method to detect deepfakes from methods unseen during training, significantly improving on several benchmark methods.