Dense Non-Rigid Structure from Motion: A Manifold Viewpoint
This work addresses the challenge of dense 3D reconstruction for non-rigid objects in computer vision, offering a more scalable and robust solution that is incremental but advances the state-of-the-art in NRSfM.
The paper tackles the problem of recovering dense 3D geometry from 2D feature correspondences for deforming objects in Non-Rigid Structure-from-Motion (NRSfM), overcoming limitations of previous methods in scalability, noise robustness, and modeling non-linear deformations, and achieves leading performance on standard benchmarks with improved accuracy.
Non-Rigid Structure-from-Motion (NRSfM) problem aims to recover 3D geometry of a deforming object from its 2D feature correspondences across multiple frames. Classical approaches to this problem assume a small number of feature points and, ignore the local non-linearities of the shape deformation, and therefore, struggles to reliably model non-linear deformations. Furthermore, available dense NRSfM algorithms are often hurdled by scalability, computations, noisy measurements and, restricted to model just global deformation. In this paper, we propose algorithms that can overcome these limitations with the previous methods and, at the same time, can recover a reliable dense 3D structure of a non-rigid object with higher accuracy. Assuming that a deforming shape is composed of a union of local linear subspace and, span a global low-rank space over multiple frames enables us to efficiently model complex non-rigid deformations. To that end, each local linear subspace is represented using Grassmannians and, the global 3D shape across multiple frames is represented using a low-rank representation. We show that our approach significantly improves accuracy, scalability, and robustness against noise. Also, our representation naturally allows for simultaneous reconstruction and clustering framework which in general is observed to be more suitable for NRSfM problems. Our method currently achieves leading performance on the standard benchmark datasets.