Synthesis in Style: Semantic Segmentation of Historical Documents using Synthetic Data
This addresses the problem of data scarcity for researchers and practitioners in historical document analysis, though it is an incremental improvement over existing synthesis methods.
The authors tackled the lack of annotated training data for historical document analysis by proposing a method to construct synthetic labeled datasets using StyleGAN, which indirectly learns semantics to generate ground truth labels. They showed that models trained on their synthetic dataset achieve better segmentation results than those using a state-of-the-art synthesis approach, with reduced human annotation effort.
One of the most pressing problems in the automated analysis of historical documents is the availability of annotated training data. The problem is that labeling samples is a time-consuming task because it requires human expertise and thus, cannot be automated well. In this work, we propose a novel method to construct synthetic labeled datasets for historical documents where no annotations are available. We train a StyleGAN model to synthesize document images that capture the core features of the original documents. While originally, the StyleGAN architecture was not intended to produce labels, it indirectly learns the underlying semantics to generate realistic images. Using our approach, we can extract the semantic information from the intermediate feature maps and use it to generate ground truth labels. To investigate if our synthetic dataset can be used to segment the text in historical documents, we use it to train multiple supervised segmentation models and evaluate their performance. We also train these models on another dataset created by a state-of-the-art synthesis approach to show that the models trained on our dataset achieve better results while requiring even less human annotation effort.