Probing-Guided Layer Selection from Self-Supervised Speech Models for Generalizable Audio Deepfake Detection
The work addresses the generalization problem in audio deepfake detection by providing a model-agnostic layer selection strategy that improves cross-domain performance with fewer parameters.
The paper proposes a two-stage method that uses lightweight probes to select informative layers from self-supervised speech models before training a classifier for audio deepfake detection. On XLS-R-300M, four selected layers achieve 4.94% EER on In-The-Wild and 5.07% cross-domain average, a 28% relative improvement over prior frozen-backbone results using all 25 layers.
Audio deepfake detection systems often fail to generalize across domains because they rely on features tied to specific attacks or recording conditions. Self-supervised speech models offer rich multi-layer representations, yet existing approaches either use a single layer or fuse all layers indiscriminately, and only reveal layer importance after training. We propose a model-agnostic, two-stage methodology that identifies informative depth zones before any task-specific model is trained. In the first stage, lightweight XGBoost probes evaluate each transformer layer's cross-domain discriminative power, producing a layer ranking. In the second stage, a compact neural classifier fuses only the selected layers through per-layer attention pooling and a shared bottleneck projection, while the backbone remains frozen. Applied across three backbones, the probing reveals two key findings. First, informative layers cluster in depth zones rather than at uniquely optimal positions: within-zone substitutions fall within multi-seed noise, while zone violations degrade performance by up to 5x. Second, the probing produces backbone-specific selections rather than a fixed layer recipe. On XLS-R-300M, four probing-selected layers with 1.34M trainable parameters achieve 4.94 +/- 0.32% equal error rate on In-The-Wild and 5.07% cross-domain average over four shared datasets, a 28% relative improvement over the best prior frozen-backbone result (Xiao and Vu, 2025) using all 25 layers with identical training data.