Towards Spectral Convergence of Locally Linear Embedding on Manifolds with Boundary
This work addresses a theoretical gap in unsupervised learning for researchers, focusing on LLE's behavior on bounded manifolds, but it is incremental as it builds on existing spectral analysis methods.
The paper tackled the problem of understanding the spectral convergence of Locally Linear Embedding (LLE) on manifolds with boundary by analyzing a differential operator's eigenvalues and eigenfunctions, showing that a regularity condition imposes consistent boundary conditions and deriving analytical limiting eigenvalues that match numerical predictions.
We study the eigenvalues and eigenfunctions of a differential operator that governs the asymptotic behavior of the unsupervised learning algorithm known as Locally Linear Embedding when a large data set is sampled from an interval or disc. In particular, the differential operator is of second order, mixed-type, and degenerates near the boundary. We show that a natural regularity condition on the eigenfunctions imposes a consistent boundary condition and use the Frobenius method to estimate pointwise behavior. We then determine the limiting sequence of eigenvalues analytically and compare them to numerical predictions. Finally, we propose a variational framework for determining eigenvalues on other compact manifolds.