Art2Real: Unfolding the Reality of Artworks via Semantically-Aware Image-to-Image Translation
This addresses the challenge of limited annotated artistic data for computer vision applications, benefiting cultural heritage and art analysis, though it is incremental as it builds on existing translation techniques.
The paper tackles the problem of applying computer vision to artworks by proposing a semantic-aware image-to-image translation method that converts artworks into photo-realistic images, reducing domain shift and improving performance on tasks like classification, detection, and segmentation.
The applicability of computer vision to real paintings and artworks has been rarely investigated, even though a vast heritage would greatly benefit from techniques which can understand and process data from the artistic domain. This is partially due to the small amount of annotated artistic data, which is not even comparable to that of natural images captured by cameras. In this paper, we propose a semantic-aware architecture which can translate artworks to photo-realistic visualizations, thus reducing the gap between visual features of artistic and realistic data. Our architecture can generate natural images by retrieving and learning details from real photos through a similarity matching strategy which leverages a weakly-supervised semantic understanding of the scene. Experimental results show that the proposed technique leads to increased realism and to a reduction in domain shift, which improves the performance of pre-trained architectures for classification, detection, and segmentation. Code is publicly available at: https://github.com/aimagelab/art2real.