Objectness Similarity: Capturing Object-Level Fidelity in 3D Scene Evaluation
This work addresses the discrepancy between automated metrics and human judgment in 3D scene evaluation, which is important for researchers and developers in computer vision and graphics, though it is incremental as it builds on existing object detection methods.
The paper tackles the problem of evaluating 3D scenes by proposing Objectness SIMilarity (OSIM), a metric that focuses on objects to better align with human perception, and shows through a user study that OSIM outperforms existing metrics in this alignment.
This paper presents Objectness SIMilarity (OSIM), a novel evaluation metric for 3D scenes that explicitly focuses on "objects," which are fundamental units of human visual perception. Existing metrics assess overall image quality, leading to discrepancies with human perception. Inspired by neuropsychological insights, we hypothesize that human recognition of 3D scenes fundamentally involves attention to individual objects. OSIM enables object-centric evaluations by leveraging an object detection model and its feature representations to quantify the "objectness" of each object in the scene. Our user study demonstrates that OSIM aligns more closely with human perception compared to existing metrics. We also analyze the characteristics of OSIM using various approaches. Moreover, we re-evaluate recent 3D reconstruction and generation models under a standardized experimental setup to clarify advancements in this field. The code is available at https://github.com/Objectness-Similarity/OSIM.