Crossing the Species Divide: Transfer Learning from Speech to Animal Sounds
This work addresses the challenge of efficient bioacoustic analysis for researchers, though it is incremental as it adapts existing speech models to a new domain.
The paper tackled the problem of applying self-supervised speech models to bioacoustic tasks, showing that models like HuBERT and WavLM generate effective representations for animal sound detection and classification, with results competitive against fine-tuned bioacoustic models.
Self-supervised speech models have demonstrated impressive performance in speech processing, but their effectiveness on non-speech data remains underexplored. We study the transfer learning capabilities of such models on bioacoustic detection and classification tasks. We show that models such as HuBERT, WavLM, and XEUS can generate rich latent representations of animal sounds across taxa. We analyze the models properties with linear probing on time-averaged representations. We then extend the approach to account for the effect of time-wise information with other downstream architectures. Finally, we study the implication of frequency range and noise on performance. Notably, our results are competitive with fine-tuned bioacoustic pre-trained models and show the impact of noise-robust pre-training setups. These findings highlight the potential of speech-based self-supervised learning as an efficient framework for advancing bioacoustic research.