An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusion
This addresses the problem of generating realistic 3D models with UV maps for applications in computer graphics and AI, offering a novel representation that simplifies 3D shape handling.
The paper tackles 3D model generation by representing 3D objects as 64x64 pixel 'Object Images' that encode geometry, appearance, and patch structures, enabling the use of image diffusion models. On the ABO dataset, it achieves point cloud FID comparable to recent 3D generative models and supports PBR material generation.
We introduce a new approach for generating realistic 3D models with UV maps through a representation termed "Object Images." This approach encapsulates surface geometry, appearance, and patch structures within a 64x64 pixel image, effectively converting complex 3D shapes into a more manageable 2D format. By doing so, we address the challenges of both geometric and semantic irregularity inherent in polygonal meshes. This method allows us to use image generation models, such as Diffusion Transformers, directly for 3D shape generation. Evaluated on the ABO dataset, our generated shapes with patch structures achieve point cloud FID comparable to recent 3D generative models, while naturally supporting PBR material generation.