Markov-Lipschitz Deep Learning
This addresses geometric issues in manifold-based representation learning and data generation for AI applications, offering a general enhancement for vector transformation networks, though it appears incremental as it builds on existing methods with a novel constraint.
The paper tackles geometric deterioration in neural networks for manifold learning and generation by proposing the Markov-Lipschitz deep learning (MLDL) framework, which imposes a locally isometric smoothness constraint via a Markov random field to enhance local geometry preservation and robustness, leading to significant advantages in experiments.
We propose a novel framework, called Markov-Lipschitz deep learning (MLDL), to tackle geometric deterioration caused by collapse, twisting, or crossing in vector-based neural network transformations for manifold-based representation learning and manifold data generation. A prior constraint, called locally isometric smoothness (LIS), is imposed across-layers and encoded into a Markov random field (MRF)-Gibbs distribution. This leads to the best possible solutions for local geometry preservation and robustness as measured by locally geometric distortion and locally bi-Lipschitz continuity. Consequently, the layer-wise vector transformations are enhanced into well-behaved, LIS-constrained metric homeomorphisms. Extensive experiments, comparisons, and ablation study demonstrate significant advantages of MLDL for manifold learning and manifold data generation. MLDL is general enough to enhance any vector transformation-based networks. The code is available at https://github.com/westlake-cairi/Markov-Lipschitz-Deep-Learning.