Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation
This work addresses a gap in evaluation for computer vision researchers working on intrinsic image decomposition, providing a new dataset and metrics to better assess albedo quality, though it is incremental as it builds on existing methods and datasets.
The paper tackles the problem of evaluating albedo recovery in intrinsic image decomposition by collecting a new dataset, Measured Albedo in the Wild (MAW), and proposing three complementary metrics (intensity, chromaticity, texture) to address limitations of the existing WHDR metric, showing that existing algorithms often improve WHDR but perform poorly on these new metrics, and finetuning algorithms on MAW significantly enhances albedo reconstruction quality.
Intrinsic image decomposition and inverse rendering are long-standing problems in computer vision. To evaluate albedo recovery, most algorithms report their quantitative performance with a mean Weighted Human Disagreement Rate (WHDR) metric on the IIW dataset. However, WHDR focuses only on relative albedo values and often fails to capture overall quality of the albedo. In order to comprehensively evaluate albedo, we collect a new dataset, Measured Albedo in the Wild (MAW), and propose three new metrics that complement WHDR: intensity, chromaticity and texture metrics. We show that existing algorithms often improve WHDR metric but perform poorly on other metrics. We then finetune different algorithms on our MAW dataset to significantly improve the quality of the reconstructed albedo both quantitatively and qualitatively. Since the proposed intensity, chromaticity, and texture metrics and the WHDR are all complementary we further introduce a relative performance measure that captures average performance. By analysing existing algorithms we show that there is significant room for improvement. Our dataset and evaluation metrics will enable researchers to develop algorithms that improve albedo reconstruction. Code and Data available at: https://measuredalbedo.github.io/