Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image
This enables efficient material acquisition for applications like computer graphics and AR, but it is incremental as it builds on existing single-image SVBRDF methods.
The paper tackles the problem of acquiring spatially-varying BRDF and normal maps from a single mobile phone image by training a CNN with physical insights and refining results with a CRF, achieving high-quality results and significant improvements over prior works.
We propose a material acquisition approach to recover the spatially-varying BRDF and normal map of a near-planar surface from a single image captured by a handheld mobile phone camera. Our method images the surface under arbitrary environment lighting with the flash turned on, thereby avoiding shadows while simultaneously capturing high-frequency specular highlights. We train a CNN to regress an SVBRDF and surface normals from this image. Our network is trained using a large-scale SVBRDF dataset and designed to incorporate physical insights for material estimation, including an in-network rendering layer to model appearance and a material classifier to provide additional supervision during training. We refine the results from the network using a dense CRF module whose terms are designed specifically for our task. The framework is trained end-to-end and produces high quality results for a variety of materials. We provide extensive ablation studies to evaluate our network on both synthetic and real data, while demonstrating significant improvements in comparisons with prior works.