Document Image Cleaning using Budget-Aware Black-Box Approximation
This work addresses the computational and financial costs of training document preprocessing systems for OCR, offering a practical solution for applications in document digitization and text recognition, though it is incremental as it builds on existing differentiable bypass methods.
The paper tackles the problem of expensive training for document image cleaning by approximating a black-box OCR engine, proposing sample selection algorithms that reduce OCR engine queries by over 90% and training time by over 60% without significant accuracy loss, and achieving a 4% improvement in word-level accuracy with only 2.5% of queries and a 32x cost reduction.
Recent work has shown that by approximating the behaviour of a non-differentiable black-box function using a neural network, the black-box can be integrated into a differentiable training pipeline for end-to-end training. This methodology is termed "differentiable bypass,'' and a successful application of this method involves training a document preprocessor to improve the performance of a black-box OCR engine. However, a good approximation of an OCR engine requires querying it for all samples throughout the training process, which can be computationally and financially expensive. Several zeroth-order optimization (ZO) algorithms have been proposed in black-box attack literature to find adversarial examples for a black-box model by computing its gradient in a query-efficient manner. However, the query complexity and convergence rate of such algorithms makes them infeasible for our problem. In this work, we propose two sample selection algorithms to train an OCR preprocessor with less than 10% of the original system's OCR engine queries, resulting in more than 60% reduction of the total training time without significant loss of accuracy. We also show an improvement of 4% in the word-level accuracy of a commercial OCR engine with only 2.5% of the total queries and a 32x reduction in monetary cost. Further, we propose a simple ranking technique to prune 30% of the document images from the training dataset without affecting the system's performance.