PhotoShape: Photorealistic Materials for Large-Scale Shape Collections
This addresses the need for photorealistic materials in 3D shape collections, which is incremental as it builds on existing data and techniques.
The paper tackles the problem of 3D shape repositories lacking photorealistic appearance by automatically assigning high-quality, realistic materials to large-scale shape collections, resulting in photorealistic, relightable 3D shapes called PhotoShapes.
Existing online 3D shape repositories contain thousands of 3D models but lack photorealistic appearance. We present an approach to automatically assign high-quality, realistic appearance models to large scale 3D shape collections. The key idea is to jointly leverage three types of online data -- shape collections, material collections, and photo collections, using the photos as reference to guide assignment of materials to shapes. By generating a large number of synthetic renderings, we train a convolutional neural network to classify materials in real photos, and employ 3D-2D alignment techniques to transfer materials to different parts of each shape model. Our system produces photorealistic, relightable, 3D shapes (PhotoShapes).