GTT-Net: Learned Generalized Trajectory Triangulation
This work addresses 3D motion reconstruction for applications like multi-instance reconstruction and event segmentation, but appears incremental as it builds on existing graph-theoretic formulations.
The paper tackles the problem of reconstructing sparse dynamic 3D geometry from non-concurrent multi-view images without global sequencing, using GTT-Net to learn spatio-temporal affinities and outperform state-of-the-art methods in accuracy and robustness for motion-capture sequences.
We present GTT-Net, a supervised learning framework for the reconstruction of sparse dynamic 3D geometry. We build on a graph-theoretic formulation of the generalized trajectory triangulation problem, where non-concurrent multi-view imaging geometry is known but global image sequencing is not provided. GTT-Net learns pairwise affinities modeling the spatio-temporal relationships among our input observations and leverages them to determine 3D geometry estimates. Experiments reconstructing 3D motion-capture sequences show GTT-Net outperforms the state of the art in terms of accuracy and robustness. Within the context of articulated motion reconstruction, our proposed architecture is 1) able to learn and enforce semantic 3D motion priors for shared training and test domains, while being 2) able to generalize its performance across different training and test domains. Moreover, GTT-Net provides a computationally streamlined framework for trajectory triangulation with applications to multi-instance reconstruction and event segmentation.