Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Models
For researchers working on few-shot audio classification, this work addresses the unexplored potential of audio-side prompts, offering a complementary approach to text-only prompting methods.
The paper proposes a framework that introduces trainable prompts into the audio encoder of Audio-Language Models to capture task-specific acoustic features, improving few-shot adaptation. Experiments across 11 datasets show that integrating this method with existing text prompt tuning yields performance improvements.
Audio-Language Models (ALMs) have shown remarkable success in zero-shot audio classification by aligning audio waveforms with text. Recent efforts to improve downstream performance focus on learning optimal text prompts. However, previous approaches focus on the text encoder, leaving the potential of learnable prompts within the audio encoder unexplored. In this paper, we propose a novel framework that introduces trainable prompts into the audio encoder to capture task-specific acoustic features. We demonstrate that integrating audio-side prompt learning with existing text-side approaches enhances few-shot adaptation. Through extensive experiments across 11 datasets show that integrating our method as a plug-and-play module alongside existing text prompt tuning generally leads to performance improvements. These findings suggest that explicitly modulating the audio representation space effectively complements text-only prompting approaches. The code is available at https://github.com/hyebin-c/aspl.