SPA-VAE: Similar-Parts-Assignment for Unsupervised 3D Point Cloud Generation
It addresses the problem of generating realistic 3D objects with repeated parts for computer vision and graphics applications, offering an incremental improvement over existing methods.
The paper tackles unsupervised parts-aware 3D point cloud generation by introducing SPA-VAE, which infers latent canonical shapes and transformations to model self-similar parts, resulting in improved modeling accuracy and generative outputs as demonstrated on ShapeNet.
This paper addresses the problem of unsupervised parts-aware point cloud generation with learned parts-based self-similarity. Our SPA-VAE infers a set of latent canonical candidate shapes for any given object, along with a set of rigid body transformations for each such candidate shape to one or more locations within the assembled object. In this way, noisy samples on the surface of, say, each leg of a table, are effectively combined to estimate a single leg prototype. When parts-based self-similarity exists in the raw data, sharing data among parts in this way confers numerous advantages: modeling accuracy, appropriately self-similar generative outputs, precise in-filling of occlusions, and model parsimony. SPA-VAE is trained end-to-end using a variational Bayesian approach which uses the Gumbel-softmax trick for the shared part assignments, along with various novel losses to provide appropriate inductive biases. Quantitative and qualitative analyses on ShapeNet demonstrate the advantage of SPA-VAE.