MatMart: Material Reconstruction of 3D Objects via Diffusion
This work addresses material estimation and generation for 3D objects, offering improved scalability and stability, but it appears incremental as it builds on existing diffusion model applications in this domain.
The paper tackles the problem of material reconstruction for 3D objects by proposing MatMart, a diffusion-based framework that achieves high-fidelity results through a two-stage process and view-material cross-attention, outperforming existing methods in experiments.
Applying diffusion models to physically-based material estimation and generation has recently gained prominence. In this paper, we propose \ttt, a novel material reconstruction framework for 3D objects, offering the following advantages. First, \ttt\ adopts a two-stage reconstruction, starting with accurate material prediction from inputs and followed by prior-guided material generation for unobserved views, yielding high-fidelity results. Second, by utilizing progressive inference alongside the proposed view-material cross-attention (VMCA), \ttt\ enables reconstruction from an arbitrary number of input images, demonstrating strong scalability and flexibility. Finally, \ttt\ achieves both material prediction and generation capabilities through end-to-end optimization of a single diffusion model, without relying on additional pre-trained models, thereby exhibiting enhanced stability across various types of objects. Extensive experiments demonstrate that \ttt\ achieves superior performance in material reconstruction compared to existing methods.