3D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representations
This work addresses limitations in 3D-aware image generation for broader 3D applications, such as computer graphics and virtual reality, by improving geometry quality and enabling re-lighting.
The paper tackles the problem of generating 3D-controllable objects from 2D images by proposing factorized representations that are view-independent and light-disentangled, enabling the extraction of albedo-textured meshes without additional labels or assumptions.
While NeRF-based 3D-aware image generation methods enable viewpoint control, limitations still remain to be adopted to various 3D applications. Due to their view-dependent and light-entangled volume representation, the 3D geometry presents unrealistic quality and the color should be re-rendered for every desired viewpoint. To broaden the 3D applicability from 3D-aware image generation to 3D-controllable object generation, we propose the factorized representations which are view-independent and light-disentangled, and training schemes with randomly sampled light conditions. We demonstrate the superiority of our method by visualizing factorized representations, re-lighted images, and albedo-textured meshes. In addition, we show that our approach improves the quality of the generated geometry via visualization and quantitative comparison. To the best of our knowledge, this is the first work that extracts albedo-textured meshes with unposed 2D images without any additional labels or assumptions.