A Proposal for Semantic Map Representation and Evaluation
This work addresses a standardization problem for researchers in semantic mapping, but it is incremental as it builds on existing methods without introducing new techniques.
The paper tackles the lack of uniform representation and standard benchmarks in semantic mapping by proposing an extensible formalism and a dataset based on real sensor data, aiming to enable direct comparison of existing methods.
Semantic mapping is the incremental process of "mapping" relevant information of the world (i.e., spatial information, temporal events, agents and actions) to a formal description supported by a reasoning engine. Current research focuses on learning the semantic of environments based on their spatial location, geometry and appearance. Many methods to tackle this problem have been proposed, but the lack of a uniform representation, as well as standard benchmarking suites, prevents their direct comparison. In this paper, we propose a standardization in the representation of semantic maps, by defining an easily extensible formalism to be used on top of metric maps of the environments. Based on this, we describe the procedure to build a dataset (based on real sensor data) for benchmarking semantic mapping techniques, also hypothesizing some possible evaluation metrics. Nevertheless, by providing a tool for the construction of a semantic map ground truth, we aim at the contribution of the scientific community in acquiring data for populating the dataset.