A Large Dataset of Historical Japanese Documents with Complex Layouts
This dataset addresses a bottleneck for researchers and practitioners in document digitization, particularly for Asian languages, though it is incremental as it builds on existing data collection methods.
The authors tackled the lack of large training datasets for historical Japanese document layout analysis by creating HJDataset, which includes over 250,000 layout element annotations and hierarchical structures, enabling baseline performance analyses with state-of-the-art models.
Deep learning-based approaches for automatic document layout analysis and content extraction have the potential to unlock rich information trapped in historical documents on a large scale. One major hurdle is the lack of large datasets for training robust models. In particular, little training data exist for Asian languages. To this end, we present HJDataset, a Large Dataset of Historical Japanese Documents with Complex Layouts. It contains over 250,000 layout element annotations of seven types. In addition to bounding boxes and masks of the content regions, it also includes the hierarchical structures and reading orders for layout elements. The dataset is constructed using a combination of human and machine efforts. A semi-rule based method is developed to extract the layout elements, and the results are checked by human inspectors. The resulting large-scale dataset is used to provide baseline performance analyses for text region detection using state-of-the-art deep learning models. And we demonstrate the usefulness of the dataset on real-world document digitization tasks. The dataset is available at https://dell-research-harvard.github.io/HJDataset/.