A Convergence Rate for Manifold Neural Networks
This work provides a theoretical foundation for geometric deep learning on manifolds, addressing convergence issues in high-dimensional data analysis, though it is incremental as it builds upon existing methods.
The authors established a convergence rate for manifold neural networks that depends on the intrinsic dimension of the manifold but is independent of the ambient dimension, building upon prior work that introduced a method for constructing such networks using spectral decomposition and a numerical scheme for implementation with finite sample points.
High-dimensional data arises in numerous applications, and the rapidly developing field of geometric deep learning seeks to develop neural network architectures to analyze such data in non-Euclidean domains, such as graphs and manifolds. Recent work by Z. Wang, L. Ruiz, and A. Ribeiro has introduced a method for constructing manifold neural networks using the spectral decomposition of the Laplace Beltrami operator. Moreover, in this work, the authors provide a numerical scheme for implementing such neural networks when the manifold is unknown and one only has access to finitely many sample points. The authors show that this scheme, which relies upon building a data-driven graph, converges to the continuum limit as the number of sample points tends to infinity. Here, we build upon this result by establishing a rate of convergence that depends on the intrinsic dimension of the manifold but is independent of the ambient dimension. We also discuss how the rate of convergence depends on the depth of the network and the number of filters used in each layer.