Low-rank Adaptation Method for Wav2vec2-based Fake Audio Detection
This work addresses the challenge of long training times and high memory consumption for researchers and practitioners in audio forensics, though it is incremental as it adapts an existing method to a specific domain.
The paper tackled the problem of high computational cost in fine-tuning large pre-trained models for fake audio detection by applying Low-rank Adaptation (LoRA) to wav2vec2, reducing trainable parameters by 198 times while achieving similar performance.
Self-supervised speech models are a rapidly developing research topic in fake audio detection. Many pre-trained models can serve as feature extractors, learning richer and higher-level speech features. However,when fine-tuning pre-trained models, there is often a challenge of excessively long training times and high memory consumption, and complete fine-tuning is also very expensive. To alleviate this problem, we apply low-rank adaptation(LoRA) to the wav2vec2 model, freezing the pre-trained model weights and injecting a trainable rank-decomposition matrix into each layer of the transformer architecture, greatly reducing the number of trainable parameters for downstream tasks. Compared with fine-tuning with Adam on the wav2vec2 model containing 317M training parameters, LoRA achieved similar performance by reducing the number of trainable parameters by 198 times.