Using Graph Neural Networks to Reconstruct Ancient Documents
This work addresses the specific challenge of ancient document reconstruction for archaeologists and historians, representing an incremental application of existing methods to a new domain.
The paper tackles the problem of reconstructing ancient documents from fragments by using a Graph Neural Network to classify spatial relationships between patches, resulting in a model that can generate partial or full reconstruction graphs.
In recent years, machine learning and deep learning approaches such as artificial neural networks have gained in popularity for the resolution of automatic puzzle resolution problems. Indeed, these methods are able to extract high-level representations from images, and then can be trained to separate matching image pieces from non-matching ones. These applications have many similarities to the problem of ancient document reconstruction from partially recovered fragments. In this work we present a solution based on a Graph Neural Network, using pairwise patch information to assign labels to edges representing the spatial relationships between pairs. This network classifies the relationship between a source and a target patch as being one of Up, Down, Left, Right or None. By doing so for all edges, our model outputs a new graph representing a reconstruction proposal. Finally, we show that our model is not only able to provide correct classifications at the edge-level, but also to generate partial or full reconstruction graphs from a set of patches.