Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder
This work addresses speech enhancement for noisy environments, offering an incremental improvement by incorporating noise awareness into a VAE framework.
The authors tackled the problem of speech enhancement by proposing a noise-aware variational autoencoder (VAE) that includes noise information during training to improve robustness, especially at low SNRs, and demonstrated it outperforms a standard VAE in distortion and generalizes better to unseen noise conditions than a supervised DNN.
Recently, a generative variational autoencoder (VAE) has been proposed for speech enhancement to model speech statistics. However, this approach only uses clean speech in the training phase, making the estimation particularly sensitive to noise presence, especially in low signal-to-noise ratios (SNRs). To increase the robustness of the VAE, we propose to include noise information in the training phase by using a noise-aware encoder trained on noisy-clean speech pairs. We evaluate our approach on real recordings of different noisy environments and acoustic conditions using two different noise datasets. We show that our proposed noise-aware VAE outperforms the standard VAE in terms of overall distortion without increasing the number of model parameters. At the same time, we demonstrate that our model is capable of generalizing to unseen noise conditions better than a supervised feedforward deep neural network (DNN). Furthermore, we demonstrate the robustness of the model performance to a reduction of the noisy-clean speech training data size.