HyWA: Hypernetwork Weight Adapting Personalized Voice Activity Detection
This addresses the problem of speaker-specific voice activation for users in speech processing, offering an incremental improvement over existing conditioning techniques.
The paper tackles personalized voice activity detection (PVAD) by using a hypernetwork to adapt weights of a standard VAD model for different speakers, resulting in improved mean average precision and simplified deployment without architectural changes.
Personalized Voice Activity Detection (PVAD) systems activate only in response to a specific target speaker by incorporating speaker embeddings from enrollment utterances. Unlike existing methods that require architectural changes, such as FiLM layers, our approach employs a hypernetwork to modify the weights of a few selected layers within a standard voice activity detection (VAD) model. This enables speaker conditioning without changing the VAD architecture, allowing the same VAD model to adapt to different speakers by updating only a small subset of the layers. We propose HyWA-PVAD, a hypernetwork weight adaptation method, and evaluate it against multiple baseline conditioning techniques. Our comparison shows consistent improvements in PVAD performance. HyWA also offers practical advantages for deployment by preserving the core VAD architecture. Our new approach improves the current conditioning techniques in two ways: i) increases the mean average precision, ii) simplifies deployment by reusing the same VAD architecture.