CU-Net: Real-Time High-Fidelity Color Upsampling for Point Clouds
This addresses a bottleneck in AR/VR and telepresence by enabling high-quality color upsampling for point clouds, though it is incremental as it builds on existing geometry upsampling methods.
The paper tackles the problem of color upsampling for point clouds, which is essential for AR/VR applications but previously overlooked, and proposes CU-Net, a deep-learning model that achieves real-time operation with high visual fidelity, colorizing nearly a million points efficiently.
Point cloud upsampling is essential for high-quality augmented reality, virtual reality, and telepresence applications, due to the capture, processing, and communication limitations of existing technologies. Although geometry upsampling to densify a point cloud's coordinates has been well studied, the upsampling of the color attributes has been largely overlooked. In this paper, we propose CU-Net, the first deep-learning point cloud color upsampling model that enables low latency and high visual fidelity operation. CU-Net achieves linear time and space complexity by leveraging a feature extractor based on sparse convolution and a color prediction module based on neural implicit function. Therefore, CU-Net is theoretically guaranteed to be more efficient than most existing methods with quadratic complexity. Experimental results demonstrate that CU-Net can colorize a photo-realistic point cloud with nearly a million points in real time, while having notably better visual performance than baselines. Besides, CU-Net can adapt to arbitrary upsampling ratios and unseen objects without retraining. Our source code is available at https://github.com/UMass-LIDS/cunet.