A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing
This addresses the need for more diverse and fair material segmentation data for scene parsing in computer vision, though it is incremental as it builds on existing methods with new data.
The authors tackled the problem of material segmentation by creating a large-scale dataset with 3.2 million dense segments across 44,560 images, which is 23 times larger than existing data, and showed that a model trained on this data outperforms state-of-the-art models across datasets and viewpoints.
A key algorithm for understanding the world is material segmentation, which assigns a label (metal, glass, etc.) to each pixel. We find that a model trained on existing data underperforms in some settings and propose to address this with a large-scale dataset of 3.2 million dense segments on 44,560 indoor and outdoor images, which is 23x more segments than existing data. Our data covers a more diverse set of scenes, objects, viewpoints and materials, and contains a more fair distribution of skin types. We show that a model trained on our data outperforms a state-of-the-art model across datasets and viewpoints. We propose a large-scale scene parsing benchmark and baseline of 0.729 per-pixel accuracy, 0.585 mean class accuracy and 0.420 mean IoU across 46 materials.