Non-linear Dimensionality Regularizer for Solving Inverse Problems
This work addresses inverse problems in machine learning, such as missing data prediction and non-rigid structure from motion, but appears incremental as it builds on the KPCA framework.
The paper tackles the problem of estimating causal factors from observations in ill-posed inverse problems by introducing a non-linear dimensionality regularization technique without pre-training, achieving state-of-the-art results on the oil flow dataset and promising outcomes on the CMU mocap dataset despite occlusions and noise.
Consider an ill-posed inverse problem of estimating causal factors from observations, one of which is known to lie near some (un- known) low-dimensional, non-linear manifold expressed by a predefined Mercer-kernel. Solving this problem requires simultaneous estimation of these factors and learning the low-dimensional representation for them. In this work, we introduce a novel non-linear dimensionality regulariza- tion technique for solving such problems without pre-training. We re-formulate Kernel-PCA as an energy minimization problem in which low dimensionality constraints are introduced as regularization terms in the energy. To the best of our knowledge, ours is the first at- tempt to create a dimensionality regularizer in the KPCA framework. Our approach relies on robustly penalizing the rank of the recovered fac- tors directly in the implicit feature space to create their low-dimensional approximations in closed form. Our approach performs robust KPCA in the presence of missing data and noise. We demonstrate state-of-the-art results on predicting missing entries in the standard oil flow dataset. Additionally, we evaluate our method on the challenging problem of Non-Rigid Structure from Motion and our approach delivers promising results on CMU mocap dataset despite the presence of significant occlusions and noise.