Building Inspection Toolkit: Unified Evaluation and Strong Baselines for Damage Recognition
This work addresses data and evaluation inconsistencies for researchers in automated building inspection, though it is incremental as it builds on existing datasets and methods.
The paper tackles the problem of incomparable results in damage recognition for building inspection by introducing a unified toolkit (bikit) that provides open-source datasets with consistent evaluation splits and metrics, and establishes strong baselines using transfer learning approaches.
In recent years, several companies and researchers have started to tackle the problem of damage recognition within the scope of automated inspection of built structures. While companies are neither willing to publish associated data nor models, researchers are facing the problem of data shortage on one hand and inconsistent dataset splitting with the absence of consistent metrics on the other hand. This leads to incomparable results. Therefore, we introduce the building inspection toolkit -- bikit -- which acts as a simple to use data hub containing relevant open-source datasets in the field of damage recognition. The datasets are enriched with evaluation splits and predefined metrics, suiting the specific task and their data distribution. For the sake of compatibility and to motivate researchers in this domain, we also provide a leaderboard and the possibility to share model weights with the community. As starting point we provide strong baselines for multi-target classification tasks utilizing extensive hyperparameter search using three transfer learning approaches for state-of-the-art algorithms. The toolkit and the leaderboard are available online.