Speech enhancement aided end-to-end multi-task learning for voice activity detection
This work addresses the problem of voice activity detection in noisy conditions for applications like speech processing, but it is incremental as it builds on existing multi-task and speech enhancement approaches.
The paper tackles robust voice activity detection (VAD) in low SNR environments by proposing an end-to-end multi-task model that jointly optimizes VAD and speech enhancement using a new objective called VAD-masked scale-invariant source-to-distortion ratio (mSI-SDR). Experimental results show the multi-task method significantly outperforms single-task VAD and mSI-SDR outperforms SI-SDR in the same setting.
Robust voice activity detection (VAD) is a challenging task in low signal-to-noise (SNR) environments. Recent studies show that speech enhancement is helpful to VAD, but the performance improvement is limited. To address this issue, here we propose a speech enhancement aided end-to-end multi-task model for VAD. The model has two decoders, one for speech enhancement and the other for VAD. The two decoders share the same encoder and speech separation network. Unlike the direct thought that takes two separated objectives for VAD and speech enhancement respectively, here we propose a new joint optimization objective -- VAD-masked scale-invariant source-to-distortion ratio (mSI-SDR). mSI-SDR uses VAD information to mask the output of the speech enhancement decoder in the training process. It makes the VAD and speech enhancement tasks jointly optimized not only at the shared encoder and separation network, but also at the objective level. It also satisfies real-time working requirement theoretically. Experimental results show that the multi-task method significantly outperforms its single-task VAD counterpart. Moreover, mSI-SDR outperforms SI-SDR in the same multi-task setting.