From Plausibility to Verifiability: Risk-Controlled Generative OCR for Vision-Language Models
This addresses deployment risks in generative OCR for vision-language models, offering a system-level risk control method that is incremental but practical.
The paper tackles the problem of generative OCR in vision-language models producing severe errors due to a misalignment between semantic plausibility and visual verifiability, proposing a Geometric Risk Controller that reduces extreme-error risk and catastrophic over-generation at predictable coverage costs.
Modern vision-language models (VLMs) can act as generative OCR engines, yet open-ended decoding can expose rare but consequential failures. We identify a core deployment misalignment in generative OCR. Autoregressive decoding favors semantic plausibility, whereas OCR requires outputs that are visually grounded and geometrically verifiable. This mismatch produces severe errors, especially over-generation and unsupported substitutions, creating deployment risk even when benchmark accuracy remains high. We therefore formulate frozen VLM OCR as a selective accept/abstain problem and propose a model-agnostic Geometric Risk Controller. The controller probes multiple structured views of the same input, applies lightweight structural screening, and accepts a transcription only when cross-view consensus and stability satisfy predefined criteria, yielding a small family of operating points. Experiments on frozen VLM backbones and standard OCR benchmarks show consistent reductions in extreme-error risk and catastrophic over-generation at predictable coverage costs. Reliable deployment of generative OCR with frozen VLMs benefits from explicit system-level risk control rather than unconstrained generation.