SIReN-VAE: Leveraging Flows and Amortized Inference for Bayesian Networks
This work addresses the need for more flexible dependency modeling in VAEs for researchers in machine learning, though it appears incremental as it builds on existing flow-based methods.
The authors tackled the problem of incorporating arbitrary dependency structures, as specified by Bayesian networks, into variational autoencoders (VAEs) to enable more complex priors and approximate posteriors, resulting in a model that shows potential in data-sparse settings on synthetic datasets.
Initial work on variational autoencoders assumed independent latent variables with simple distributions. Subsequent work has explored incorporating more complex distributions and dependency structures: including normalizing flows in the encoder network allows latent variables to entangle non-linearly, creating a richer class of distributions for the approximate posterior, and stacking layers of latent variables allows more complex priors to be specified for the generative model. This work explores incorporating arbitrary dependency structures, as specified by Bayesian networks, into VAEs. This is achieved by extending both the prior and inference network with graphical residual flows - residual flows that encode conditional independence by masking the weight matrices of the flow's residual blocks. We compare our model's performance on several synthetic datasets and show its potential in data-sparse settings.