IDEA-Net: Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment
This work addresses the challenge of 3D motion data acquisition for applications requiring smooth point cloud sequences, though it appears incremental as it builds on existing interpolation methods with novel modules.
The paper tackles the problem of temporally interpolating dynamic 3D point clouds with large non-rigid deformation by proposing IDEA-Net, an end-to-end deep learning framework that estimates point-wise trajectories and achieves large improvement over state-of-the-art methods both quantitatively and visually.
This paper investigates the problem of temporally interpolating dynamic 3D point clouds with large non-rigid deformation. We formulate the problem as estimation of point-wise trajectories (i.e., smooth curves) and further reason that temporal irregularity and under-sampling are two major challenges. To tackle the challenges, we propose IDEA-Net, an end-to-end deep learning framework, which disentangles the problem under the assistance of the explicitly learned temporal consistency. Specifically, we propose a temporal consistency learning module to align two consecutive point cloud frames point-wisely, based on which we can employ linear interpolation to obtain coarse trajectories/in-between frames. To compensate the high-order nonlinear components of trajectories, we apply aligned feature embeddings that encode local geometry properties to regress point-wise increments, which are combined with the coarse estimations. We demonstrate the effectiveness of our method on various point cloud sequences and observe large improvement over state-of-the-art methods both quantitatively and visually. Our framework can bring benefits to 3D motion data acquisition. The source code is publicly available at https://github.com/ZENGYIMING-EAMON/IDEA-Net.git.