Digitizing Historical Balance Sheet Data: A Practitioner's Guide
This provides a cost-efficient tool for economic historians to accurately extract structured information from large-scale historical micro-data.
The paper tackles the problem of high error rates in off-the-shelf OCR for digitizing historical balance sheet data by augmenting it with pre- and post-processing methods, resulting in a dramatic increase in success rate and the introduction of a Python package called quipucamayoc.
This paper discusses how to successfully digitize large-scale historical micro-data by augmenting optical character recognition (OCR) engines with pre- and post-processing methods. Although OCR software has improved dramatically in recent years due to improvements in machine learning, off-the-shelf OCR applications still present high error rates which limit their applications for accurate extraction of structured information. Complementing OCR with additional methods can however dramatically increase its success rate, making it a powerful and cost-efficient tool for economic historians. This paper showcases these methods and explains why they are useful. We apply them against two large balance sheet datasets and introduce quipucamayoc, a Python package containing these methods in a unified framework.