Semi-Implicit Stochastic Recurrent Neural Networks
This work addresses the problem of modeling complex sequential data for machine learning researchers, but it is incremental as it builds on existing stochastic recurrent neural networks with a specific variational inference improvement.
The paper tackled the limited expressive power of stochastic recurrent neural networks due to Gaussian latent variable assumptions by developing a semi-implicit stochastic recurrent neural network (SIS-RNN) using semi-implicit variational inference, resulting in outperformance over existing methods in experiments on real-world datasets.
Stochastic recurrent neural networks with latent random variables of complex dependency structures have shown to be more successful in modeling sequential data than deterministic deep models. However, the majority of existing methods have limited expressive power due to the Gaussian assumption of latent variables. In this paper, we advocate learning implicit latent representations using semi-implicit variational inference to further increase model flexibility. Semi-implicit stochastic recurrent neural network(SIS-RNN) is developed to enrich inferred model posteriors that may have no analytic density functions, as long as independent random samples can be generated via reparameterization. Extensive experiments in different tasks on real-world datasets show that SIS-RNN outperforms the existing methods.