Diffusion-based Semi-supervised Spectral Algorithm for Regression on Manifolds
This addresses regression problems for high-dimensional data with manifold structures, offering an incremental improvement over traditional spectral methods.
The paper tackles regression on high-dimensional data embedded in lower-dimensional manifolds by introducing a diffusion-based spectral algorithm that uses graph Laplacian approximation and a semi-supervised framework, achieving a convergence rate dependent only on the intrinsic manifold dimension to avoid the curse of dimensionality.
We introduce a novel diffusion-based spectral algorithm to tackle regression analysis on high-dimensional data, particularly data embedded within lower-dimensional manifolds. Traditional spectral algorithms often fall short in such contexts, primarily due to the reliance on predetermined kernel functions, which inadequately address the complex structures inherent in manifold-based data. By employing graph Laplacian approximation, our method uses the local estimation property of heat kernel, offering an adaptive, data-driven approach to overcome this obstacle. Another distinct advantage of our algorithm lies in its semi-supervised learning framework, enabling it to fully use the additional unlabeled data. This ability enhances the performance by allowing the algorithm to dig the spectrum and curvature of the data manifold, providing a more comprehensive understanding of the dataset. Moreover, our algorithm performs in an entirely data-driven manner, operating directly within the intrinsic manifold structure of the data, without requiring any predefined manifold information. We provide a convergence analysis of our algorithm. Our findings reveal that the algorithm achieves a convergence rate that depends solely on the intrinsic dimension of the underlying manifold, thereby avoiding the curse of dimensionality associated with the higher ambient dimension.