Material Anything: Generating Materials for Any 3D Object via Diffusion
This addresses the challenge of automated material generation for 3D objects, which is incremental as it builds on diffusion models with specific enhancements.
The paper tackles the problem of generating physically-based materials for 3D objects by introducing Material Anything, a fully-automated diffusion framework that outperforms existing methods across diverse object categories and lighting conditions.
We present Material Anything, a fully-automated, unified diffusion framework designed to generate physically-based materials for 3D objects. Unlike existing methods that rely on complex pipelines or case-specific optimizations, Material Anything offers a robust, end-to-end solution adaptable to objects under diverse lighting conditions. Our approach leverages a pre-trained image diffusion model, enhanced with a triple-head architecture and rendering loss to improve stability and material quality. Additionally, we introduce confidence masks as a dynamic switcher within the diffusion model, enabling it to effectively handle both textured and texture-less objects across varying lighting conditions. By employing a progressive material generation strategy guided by these confidence masks, along with a UV-space material refiner, our method ensures consistent, UV-ready material outputs. Extensive experiments demonstrate our approach outperforms existing methods across a wide range of object categories and lighting conditions.