JigsawNet: Shredded Image Reassembly using Convolutional Neural Network and Loop-based Composition
This work addresses the domain-specific problem of image reassembly for applications like forensics or archaeology, representing an incremental advancement by combining neural networks with improved global search strategies.
The paper tackles the problem of reassembling arbitrarily shredded images by introducing a deep convolutional neural network for pairwise stitching compatibility and loop-based global composition algorithms, achieving significant performance improvements over existing methods on challenging puzzles with many fragments.
This paper proposes a novel algorithm to reassemble an arbitrarily shredded image to its original status. Existing reassembly pipelines commonly consist of a local matching stage and a global compositions stage. In the local stage, a key challenge in fragment reassembly is to reliably compute and identify correct pairwise matching, for which most existing algorithms use handcrafted features, and hence, cannot reliably handle complicated puzzles. We build a deep convolutional neural network to detect the compatibility of a pairwise stitching, and use it to prune computed pairwise matches. To improve the network efficiency and accuracy, we transfer the calculation of CNN to the stitching region and apply a boost training strategy. In the global composition stage, we modify the commonly adopted greedy edge selection strategies to two new loop closure based searching algorithms. Extensive experiments show that our algorithm significantly outperforms existing methods on solving various puzzles, especially those challenging ones with many fragment pieces.