Learning 3D Representations from Procedural 3D Programs
This addresses the scalability and copyright issues in 3D data acquisition for machine learning applications, though it is incremental in leveraging existing self-supervised methods.
The paper tackles the challenge of acquiring 3D representations by learning from procedurally generated 3D shapes instead of semantically recognizable models, achieving performance on par with state-of-the-art methods across tasks like shape classification and part segmentation.
Self-supervised learning has emerged as a promising approach for acquiring transferable 3D representations from unlabeled 3D point clouds. Unlike 2D images, which are widely accessible, acquiring 3D assets requires specialized expertise or professional 3D scanning equipment, making it difficult to scale and raising copyright concerns. To address these challenges, we propose learning 3D representations from procedural 3D programs that automatically generate 3D shapes using simple primitives and augmentations. Remarkably, despite lacking semantic content, the 3D representations learned from the procedurally generated 3D shapes perform on par with state-of-the-art representations learned from semantically recognizable 3D models (e.g., airplanes) across various downstream 3D tasks, including shape classification, part segmentation, and masked point cloud completion. We provide a detailed analysis on factors that make a good 3D procedural program. Extensive experiments further suggest that current self-supervised learning methods on point clouds do not rely on the semantics of 3D shapes, shedding light on the nature of 3D representations learned.