Learning Globally Smooth Functions on Manifolds
This addresses the problem of improving generalization and stability in machine learning for applications involving low-dimensional data structures, but it appears incremental as it builds on existing techniques.
The paper tackled the problem of learning globally smooth functions on manifolds by combining semi-infinite constrained learning and manifold regularization, showing that it leads to a practical algorithm that estimates the Lipschitz constant and achieves smooth solutions, with experiments demonstrating advantages over existing methods.
Smoothness and low dimensional structures play central roles in improving generalization and stability in learning and statistics. This work combines techniques from semi-infinite constrained learning and manifold regularization to learn representations that are globally smooth on a manifold. To do so, it shows that under typical conditions the problem of learning a Lipschitz continuous function on a manifold is equivalent to a dynamically weighted manifold regularization problem. This observation leads to a practical algorithm based on a weighted Laplacian penalty whose weights are adapted using stochastic gradient techniques. It is shown that under mild conditions, this method estimates the Lipschitz constant of the solution, learning a globally smooth solution as a byproduct. Experiments on real world data illustrate the advantages of the proposed method relative to existing alternatives.