IPDreamer: Appearance-Controllable 3D Object Generation with Complex Image Prompts
This addresses the challenge of appearance control in 3D generation for applications like virtual reality and design, representing an incremental improvement over existing methods.
The paper tackles the problem of unpredictable appearance in 3D object generation from text or single images by introducing IPDreamer, which captures appearance features from complex image prompts to enable high-fidelity, appearance-controllable 3D object generation, with experiments showing consistent high-quality results.
Recent advances in 3D generation have been remarkable, with methods such as DreamFusion leveraging large-scale text-to-image diffusion-based models to guide 3D object generation. These methods enable the synthesis of detailed and photorealistic textured objects. However, the appearance of 3D objects produced by such text-to-3D models is often unpredictable, and it is hard for single-image-to-3D methods to deal with images lacking a clear subject, complicating the generation of appearance-controllable 3D objects from complex images. To address these challenges, we present IPDreamer, a novel method that captures intricate appearance features from complex $\textbf{I}$mage $\textbf{P}$rompts and aligns the synthesized 3D object with these extracted features, enabling high-fidelity, appearance-controllable 3D object generation. Our experiments demonstrate that IPDreamer consistently generates high-quality 3D objects that align with both the textual and complex image prompts, highlighting its promising capability in appearance-controlled, complex 3D object generation. Our code is available at https://github.com/zengbohan0217/IPDreamer.