Accurate 3D Reconstruction of Dynamic Scenes from Monocular Image Sequences with Severe Occlusions
This addresses the problem of accurate 3D reconstruction in dynamic scenes with occlusions for computer vision applications, but it is incremental as it builds on existing NRSfM methods.
The paper tackles dense 3D reconstruction from monocular videos with severe occlusions by integrating a shape prior into a variational optimization framework, showing significant outperformance over state-of-the-art methods in experiments on synthetic and real data.
The paper introduces an accurate solution to dense orthographic Non-Rigid Structure from Motion (NRSfM) in scenarios with severe occlusions or, likewise, inaccurate correspondences. We integrate a shape prior term into variational optimisation framework. It allows to penalize irregularities of the time-varying structure on the per-pixel level if correspondence quality indicator such as an occlusion tensor is available. We make a realistic assumption that several non-occluded views of the scene are sufficient to estimate an initial shape prior, though the entire observed scene may exhibit non-rigid deformations. Experiments on synthetic and real image data show that the proposed framework significantly outperforms state of the art methods for correspondence establishment in combination with the state of the art NRSfM methods. Together with the profound insights into optimisation methods, implementation details for heterogeneous platforms are provided.