Be your own Benchmark: No-Reference Trajectory Metric on Registered Point Clouds
This provides a trajectory benchmarking tool for setups using 3D sensing technologies, such as small-scale outdoor mapping, but it is incremental as it builds on existing metrics.
The paper tackles the problem of assessing trajectory quality without ground truth poses, proposing a no-reference metric called Mutually Orthogonal Metric (MOM) that estimates map quality from registered point clouds and correlates strongly with Relative Pose Error, with statistical confirmation in synthetic environments.
This paper addresses the problem of assessing trajectory quality in conditions when no ground truth poses are available or when their accuracy is not enough for the specific task - for example, small-scale mapping in outdoor scenes. In our work, we propose a no-reference metric, Mutually Orthogonal Metric (MOM), that estimates the quality of the map from registered point clouds via the trajectory poses. MOM strongly correlates with full-reference trajectory metric Relative Pose Error, making it a trajectory benchmarking tool on setups where 3D sensing technologies are employed. We provide a mathematical foundation for such correlation and confirm it statistically in synthetic environments. Furthermore, since our metric uses a subset of points from mutually orthogonal surfaces, we provide an algorithm for the extraction of such subset and evaluate its performance in synthetic CARLA environment and on KITTI dataset. The code of the proposed metric is publicly available as pip-package.