A Benchmark of Dynamical Variational Autoencoders applied to Speech Spectrogram Modeling
This work provides a comparative evaluation of DVAEs for speech processing, which is incremental as it builds on existing models without introducing new methods.
The authors benchmarked six Dynamical Variational Autoencoder (DVAE) models on speech spectrogram modeling, showing their high potential for speech analysis-resynthesis tasks.
The Variational Autoencoder (VAE) is a powerful deep generative model that is now extensively used to represent high-dimensional complex data via a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, input data vectors are processed independently. In recent years, a series of papers have presented different extensions of the VAE to process sequential data, that not only model the latent space, but also model the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks. We recently performed a comprehensive review of those models and unified them into a general class called Dynamical Variational Autoencoders (DVAEs). In the present paper, we present the results of an experimental benchmark comparing six of those DVAE models on the speech analysis-resynthesis task, as an illustration of the high potential of DVAEs for speech modeling.