A metric for evaluating 3D reconstruction and mapping performance with no ground truthing
This enables more flexible evaluation of 3D mapping methods, particularly benefiting applications where ground truth collection is difficult.
The paper tackles the problem of evaluating 3D reconstruction and mapping performance without ground truth data by proposing a metric called dense map posterior (DMP). Results show DMP provides similar evaluation capability to ground truth-based metrics.
It is not easy when evaluating 3D mapping performance because existing metrics require ground truth data that can only be collected with special instruments. In this paper, we propose a metric, dense map posterior (DMP), for this evaluation. It can work without any ground truth data. Instead, it calculates a comparable value, reflecting a map posterior probability, from dense point cloud observations. In our experiments, the proposed DMP is benchmarked against ground truth-based metrics. Results show that DMP can provide a similar evaluation capability. The proposed metric makes evaluating different methods more flexible and opens many new possibilities, such as self-supervised methods and more available datasets.