ARTeFACT: Benchmarking Segmentation Models on Diverse Analogue Media Damage
This addresses the challenge of reliable damage detection for cultural heritage preservation, though it is incremental as it focuses on benchmarking and dataset creation rather than a novel method.
The paper tackles the problem of robust damage detection in analogue media for cultural heritage preservation, showing that existing machine learning models fail to predict damage locations reliably, and introduces the ARTeFACT dataset with over 11,000 annotations across 15 damage types to benchmark segmentation models, revealing limitations in generalization across media types.
Accurately detecting and classifying damage in analogue media such as paintings, photographs, textiles, mosaics, and frescoes is essential for cultural heritage preservation. While machine learning models excel in correcting degradation if the damage operator is known a priori, we show that they fail to robustly predict where the damage is even after supervised training; thus, reliable damage detection remains a challenge. Motivated by this, we introduce ARTeFACT, a dataset for damage detection in diverse types analogue media, with over 11,000 annotations covering 15 kinds of damage across various subjects, media, and historical provenance. Furthermore, we contribute human-verified text prompts describing the semantic contents of the images, and derive additional textual descriptions of the annotated damage. We evaluate CNN, Transformer, diffusion-based segmentation models, and foundation vision models in zero-shot, supervised, unsupervised and text-guided settings, revealing their limitations in generalising across media types. Our dataset is available at $\href{https://daniela997.github.io/ARTeFACT/}{https://daniela997.github.io/ARTeFACT/}$ as the first-of-its-kind benchmark for analogue media damage detection and restoration.