Contrastive Learning of General-Purpose Audio Representations
This work addresses the need for effective audio representations across domains like speech and music, but it is incremental as it builds on existing contrastive learning methods.
The authors tackled the problem of learning general-purpose audio representations by introducing COLA, a self-supervised contrastive learning approach that pre-trains on Audioset and transfers to 9 diverse classification tasks, showing significant performance improvements over previous self-supervised systems.
We introduce COLA, a self-supervised pre-training approach for learning a general-purpose representation of audio. Our approach is based on contrastive learning: it learns a representation which assigns high similarity to audio segments extracted from the same recording while assigning lower similarity to segments from different recordings. We build on top of recent advances in contrastive learning for computer vision and reinforcement learning to design a lightweight, easy-to-implement self-supervised model of audio. We pre-train embeddings on the large-scale Audioset database and transfer these representations to 9 diverse classification tasks, including speech, music, animal sounds, and acoustic scenes. We show that despite its simplicity, our method significantly outperforms previous self-supervised systems. We furthermore conduct ablation studies to identify key design choices and release a library to pre-train and fine-tune COLA models.