Bayesian Manifold Learning: The Locally Linear Latent Variable Model (LL-LVM)
This provides a probabilistic framework for manifold learning, facilitating tasks like neighborhood evaluation and dimensionality selection, but it is incremental as it builds on existing non-probabilistic methods.
The paper tackles the problem of non-linear manifold discovery by introducing the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model that enables variational optimization of manifold coordinates and locally linear maps, encapsulating local-geometry preserving intuitions from methods like LLE.
We introduce the Locally Linear Latent Variable Model (LL-LVM), a probabilistic model for non-linear manifold discovery that describes a joint distribution over observations, their manifold coordinates and locally linear maps conditioned on a set of neighbourhood relationships. The model allows straightforward variational optimisation of the posterior distribution on coordinates and locally linear maps from the latent space to the observation space given the data. Thus, the LL-LVM encapsulates the local-geometry preserving intuitions that underlie non-probabilistic methods such as locally linear embedding (LLE). Its probabilistic semantics make it easy to evaluate the quality of hypothesised neighbourhood relationships, select the intrinsic dimensionality of the manifold, construct out-of-sample extensions and to combine the manifold model with additional probabilistic models that capture the structure of coordinates within the manifold.