Can You Read Me Now? Content Aware Rectification using Angle Supervision
This addresses the challenge of document rectification for OCR systems in real-world smartphone-captured images, offering a novel content-aware approach that is incremental over prior boundary-focused methods.
The paper tackles the problem of rectifying photographed documents with folds and crumples to improve OCR accuracy by introducing CREASE, a learned method that uses document content and word orientation as hints, achieving superior results in OCR accuracy, geometric error, and visual similarity compared to previous approaches.
The ubiquity of smartphone cameras has led to more and more documents being captured by cameras rather than scanned. Unlike flatbed scanners, photographed documents are often folded and crumpled, resulting in large local variance in text structure. The problem of document rectification is fundamental to the Optical Character Recognition (OCR) process on documents, and its ability to overcome geometric distortions significantly affects recognition accuracy. Despite the great progress in recent OCR systems, most still rely on a pre-process that ensures the text lines are straight and axis aligned. Recent works have tackled the problem of rectifying document images taken in-the-wild using various supervision signals and alignment means. However, they focused on global features that can be extracted from the document's boundaries, ignoring various signals that could be obtained from the document's content. We present CREASE: Content Aware Rectification using Angle Supervision, the first learned method for document rectification that relies on the document's content, the location of the words and specifically their orientation, as hints to assist in the rectification process. We utilize a novel pixel-wise angle regression approach and a curvature estimation side-task for optimizing our rectification model. Our method surpasses previous approaches in terms of OCR accuracy, geometric error and visual similarity.