Bayesian neural networks and dimensionality reduction
This work addresses uncertainty and interpretability issues in dimensionality reduction for machine learning practitioners, but it is incremental as it builds on existing methods and highlights limitations rather than providing a complete solution.
The authors tackled the problem of inadequate uncertainty quantification and instability in variational auto-encoders (VAEs) for dimensionality reduction by deploying Markov chain Monte Carlo (MCMC) sampling for Bayesian inference in neural networks with latent variables, but found that current MCMC schemes face fundamental challenges in such models.
In conducting non-linear dimensionality reduction and feature learning, it is common to suppose that the data lie near a lower-dimensional manifold. A class of model-based approaches for such problems includes latent variables in an unknown non-linear regression function; this includes Gaussian process latent variable models and variational auto-encoders (VAEs) as special cases. VAEs are artificial neural networks (ANNs) that employ approximations to make computation tractable; however, current implementations lack adequate uncertainty quantification in estimating the parameters, predictive densities, and lower-dimensional subspace, and can be unstable and lack interpretability in practice. We attempt to solve these problems by deploying Markov chain Monte Carlo sampling algorithms (MCMC) for Bayesian inference in ANN models with latent variables. We address issues of identifiability by imposing constraints on the ANN parameters as well as by using anchor points. This is demonstrated on simulated and real data examples. We find that current MCMC sampling schemes face fundamental challenges in neural networks involving latent variables, motivating new research directions.