Constructing Image-Text Pair Dataset from Books
This work addresses the need for automated dataset construction from digital archives to support machine learning applications, though it is incremental as it builds on existing tools.
The authors tackled the problem of extracting image-text pairs from digitized books by designing a pipeline with OCR, object detection, and layout analysis, and demonstrated its effectiveness in image-text retrieval and insight extraction on old photo books.
Digital archiving is becoming widespread owing to its effectiveness in protecting valuable books and providing knowledge to many people electronically. In this paper, we propose a novel approach to leverage digital archives for machine learning. If we can fully utilize such digitized data, machine learning has the potential to uncover unknown insights and ultimately acquire knowledge autonomously, just like humans read books. As a first step, we design a dataset construction pipeline comprising an optical character reader (OCR), an object detector, and a layout analyzer for the autonomous extraction of image-text pairs. In our experiments, we apply our pipeline on old photo books to construct an image-text pair dataset, showing its effectiveness in image-text retrieval and insight extraction.