Towards Lightweight Adaptation of Speech Enhancement Models in Real-World Environments
This work provides a practical solution for on-device adaptation of speech enhancement models, benefiting users who require robust performance in dynamically changing acoustic conditions.
This paper addresses the challenge of adapting speech enhancement models to new noise conditions in real-world environments by proposing a lightweight framework. The method updates less than 1% of the base model's parameters, achieving an average 1.51 dB SI-SDR improvement within 20 updates per scene across 111 diverse environments.
Recent studies have shown that post-deployment adaptation can improve the robustness of speech enhancement models in unseen noise conditions. However, existing methods often incur prohibitive computational and memory costs, limiting their suitability for on-device deployment. In this work, we investigate model adaptation in realistic settings with dynamic acoustic scene changes and propose a lightweight framework that augments a frozen backbone with low-rank adapters updated via self-supervised training. Experiments on sequential scene evaluations spanning 111 environments across 37 noise types and three signal-to-noise ratio ranges, including the challenging [-8, 0] dB range, show that our method updates fewer than 1% of the base model's parameters while achieving an average 1.51 dB SI-SDR improvement within only 20 updates per scene. Compared to state-of-the-art approaches, our framework achieves competitive or superior perceptual quality with smoother and more stable convergence, demonstrating its practicality for lightweight on-device adaptation of speech enhancement models under real-world acoustic conditions.