DiffPMAE: Diffusion Masked Autoencoders for Point Cloud Reconstruction
This addresses bandwidth and storage challenges in point cloud data for interactive services, though it appears incremental as it builds on existing self-supervised and diffusion techniques.
The paper tackles the problem of high-fidelity point cloud reconstruction for applications like streaming and the Metaverse, proposing DiffPMAE, which combines masked auto-encoding and diffusion models to achieve state-of-the-art performance on datasets like ShapeNet-55 and ModelNet with over 60,000 objects.
Point cloud streaming is increasingly getting popular, evolving into the norm for interactive service delivery and the future Metaverse. However, the substantial volume of data associated with point clouds presents numerous challenges, particularly in terms of high bandwidth consumption and large storage capacity. Despite various solutions proposed thus far, with a focus on point cloud compression, upsampling, and completion, these reconstruction-related methods continue to fall short in delivering high fidelity point cloud output. As a solution, in DiffPMAE, we propose an effective point cloud reconstruction architecture. Inspired by self-supervised learning concepts, we combine Masked Auto-Encoding and Diffusion Model mechanism to remotely reconstruct point cloud data. By the nature of this reconstruction process, DiffPMAE can be extended to many related downstream tasks including point cloud compression, upsampling and completion. Leveraging ShapeNet-55 and ModelNet datasets with over 60000 objects, we validate the performance of DiffPMAE exceeding many state-of-the-art methods in-terms of auto-encoding and downstream tasks considered.