No Pressure! Addressing the Problem of Local Minima in Manifold Learning Algorithms
This addresses a specific bottleneck in nonlinear embedding methods for data visualization, but it is incremental as it builds on existing techniques.
The paper tackles the problem of local minima in manifold learning algorithms, which leads to poor embedding quality, by proposing an extension that identifies and temporarily uses an extra dimension to improve the objective function value.
Nonlinear embedding manifold learning methods provide invaluable visual insights into the structure of high-dimensional data. However, due to a complicated nonconvex objective function, these methods can easily get stuck in local minima and their embedding quality can be poor. We propose a natural extension to several manifold learning methods aimed at identifying pressured points, i.e. points stuck in poor local minima and have poor embedding quality. We show that the objective function can be decreased by temporarily allowing these points to make use of an extra dimension in the embedding space. Our method is able to improve the objective function value of existing methods even after they get stuck in a poor local minimum.