A Multi-modal Registration and Visualization Software Tool for Artworks using CraquelureNet
This addresses the problem of pixel-wise comparison for art investigations by automating multi-modal image registration, though it appears incremental as it builds on existing registration methods with a domain-specific adaptation.
The researchers developed a software tool that automatically registers multi-modal images of artworks using a convolutional neural network to extract crack structure features, achieving effective registration performance with short inference times on historical paintings and demonstrating transferability to historical prints.
For art investigations of paintings, multiple imaging technologies, such as visual light photography, infrared reflectography, ultraviolet fluorescence photography, and x-radiography are often used. For a pixel-wise comparison, the multi-modal images have to be registered. We present a registration and visualization software tool, that embeds a convolutional neural network to extract cross-modal features of the crack structures in historical paintings for automatic registration. The graphical user interface processes the user's input to configure the registration parameters and to interactively adapt the image views with the registered pair and image overlays, such as by individual or synchronized zoom or movements of the views. In the evaluation, we qualitatively and quantitatively show the effectiveness of our software tool in terms of registration performance and short inference time on multi-modal paintings and its transferability by applying our method to historical prints.