FSA-GRPO: Teaching Auditory LLMs to Use Few-shot Demonstrations
This work addresses the lack of explicit few-shot training for auditory LLMs, enabling better adaptation to low-resource tasks, though it is an incremental improvement over existing RL-based post-training methods.
FSA-GRPO uses reinforcement learning to train auditory LLMs to better leverage few-shot demonstrations, improving performance on children's speech recognition, speech translation, and audio understanding without in-domain training data.
Few-shot prompting provides an effective way to adapt auditory large language models to low-resource tasks such as children's speech recognition. However, most auditory large language models are not explicitly trained to perform inference in this demonstration-conditioned format, limiting the extent to which they can benefit from few-shot prompting. To address this limitation, we introduce Few-Shot Aware GRPO (FSA-GRPO), an RL-based post-training recipe that uses a specially designed reward to encourage the model to leverage few-shot demonstrations, thereby strengthening its few-shot adaptation ability. Notably, training with only high-resource adult ASR data improves the model's general few-shot adaptation ability, yielding gains not only in children's speech recognition but also in speech translation and audio understanding. We further study data selection and auxiliary reward weighting to identify an effective training recipe. Our experiments show that when in-domain data are unavailable or cannot be used for training, FSA-GRPO is more effective than direct tuning on related out-of-domain data.