Self-expressive Dictionary Learning for Dynamic 3D Reconstruction
This addresses a specific challenge in computer vision for dynamic 3D reconstruction, offering a novel method for unsynchronized multi-camera setups, but it appears incremental as it builds on existing compressed sensing and dictionary learning techniques.
The paper tackles the problem of sparse 3D reconstruction of dynamic objects from multiple unsynchronized video cameras without known temporal overlap, proposing a compressed sensing framework based on dictionary learning that recovers structure and sequencing information, with experimental validation on synthetic and real data.
We target the problem of sparse 3D reconstruction of dynamic objects observed by multiple unsynchronized video cameras with unknown temporal overlap. To this end, we develop a framework to recover the unknown structure without sequencing information across video sequences. Our proposed compressed sensing framework poses the estimation of 3D structure as the problem of dictionary learning, where the dictionary is defined as an aggregation of the temporally varying 3D structures. Given the smooth motion of dynamic objects, we observe any element in the dictionary can be well approximated by a sparse linear combination of other elements in the same dictionary (i. e. self-expression). Moreover, the sparse coefficients describing a locally linear 3D structural interpolation reveal the local sequencing information. Our formulation optimizes a biconvex cost function that leverages a compressed sensing formulation and enforces both structural dependency coherence across video streams, as well as motion smoothness across estimates from common video sources. We further analyze the reconstructability of our approach under different capture scenarios, and its comparison and relation to existing methods. Experimental results on large amounts of synthetic data as well as real imagery demonstrate the effectiveness of our approach.