OpenMaterial: A Large-scale Dataset of Complex Materials for 3D Reconstruction
This addresses the lack of benchmark datasets for material-dependent 3D reconstruction, enabling more robust techniques for real-world applications, though it is incremental as it builds on existing datasets and methods.
The authors tackled the challenge of 3D reconstruction for objects with complex optical properties like metals and glass by introducing OpenMaterial, a large-scale semi-synthetic dataset of 1,001 objects across 295 materials under 714 lighting conditions, which they used to evaluate 11 state-of-the-art methods and establish the first extensive benchmark for material-aware 3D reconstruction.
Recent advances in deep learning, such as neural radiance fields and implicit neural representations, have significantly advanced 3D reconstruction. However, accurately reconstructing objects with complex optical properties, such as metals, glass, and plastics, remains challenging due to the breakdown of multi-view color consistency in the presence of specular reflections, refractions, and transparency. This limitation is further exacerbated by the lack of benchmark datasets that explicitly model material-dependent light transport. To address this, we introduce OpenMaterial, a large-scale semi-synthetic dataset for benchmarking material-aware 3D reconstruction. It comprises 1,001 objects spanning 295 distinct materials, including conductors, dielectrics, plastics, and their roughened variants, captured under 714 diverse lighting conditions. By integrating lab-measured Index of Refraction (IOR) spectra, OpenMaterial enables the generation of high-fidelity multi-view images that accurately simulate complex light-matter interactions. It provides multi-view images, 3D shape models, camera poses, depth maps, and object masks, establishing the first extensive benchmark for evaluating 3D reconstruction on challenging materials. We evaluate 11 state-of-the-art methods for 3D reconstruction and novel view synthesis, conducting ablation studies to assess the impact of material type, shape complexity, and illumination on reconstruction performance. Our results indicate that OpenMaterial provides a strong and fair basis for developing more robust, physically-informed 3D reconstruction techniques to better handle real-world optical complexities.