Learning A Locally Unified 3D Point Cloud for View Synthesis
This addresses view synthesis for 3D scenes, offering incremental improvements in accuracy and detail preservation.
The paper tackles the problem of 3D view synthesis from sparse source views by proposing a method that learns a locally unified 3D point cloud, resulting in an average PSNR improvement of over 4 dB compared to state-of-the-art methods.
In this paper, we explore the problem of 3D point cloud representation-based view synthesis from a set of sparse source views. To tackle this challenging problem, we propose a new deep learning-based view synthesis paradigm that learns a locally unified 3D point cloud from source views. Specifically, we first construct sub-point clouds by projecting source views to 3D space based on their depth maps. Then, we learn the locally unified 3D point cloud by adaptively fusing points at a local neighborhood defined on the union of the sub-point clouds. Besides, we also propose a 3D geometry-guided image restoration module to fill the holes and recover high-frequency details of the rendered novel views. Experimental results on three benchmark datasets demonstrate that our method can improve the average PSNR by more than 4 dB while preserving more accurate visual details, compared with state-of-the-art view synthesis methods.