How Do Document Parsers Break? Auditing Structural Vulnerability in Document Intelligence
For researchers and practitioners in document intelligence, this work provides a structure-aware auditing method to evaluate robustness of DLA pipelines, shifting from footprint-based stress testing to vulnerability auditing.
The paper identifies a 'Footprint Bias' in document layout analysis robustness evaluation and proposes a lightweight auditing framework that uses block-level structural loss rate and exposure descriptors to analyze structural vulnerabilities. Experiments on MinerU and PP-StructureV3 show that block-level structural loss rate aligns closely with OCR instability (R²=0.727/0.916), while small targeted probes cause downstream QA/retrieval degradation comparable to larger perturbations.
Document Layout Analysis (DLA) pipelines provide structured page representations for retrieval-augmented generation, long-document question answering, and other document intelligence systems, yet their robustness evaluation remains largely area-centric. We identify this Footprint Bias and propose a lightweight output-level auditing framework that decouples probe construction, policy-driven targeting, and structure-aware diagnosis. The framework combines Block-level Structural Loss Rate (B-SLR), granularity-aware exposure descriptors, and pathway attribution to analyze where perturbations interact with layout structure and how failures propagate. Across MinerU and PP-StructureV3 on 1,000 pages, affected area weakly tracks perturbation-induced OCR instability (R^2=0.384/0.110), whereas B-SLR aligns much more closely with it (R^2=0.727/0.916). Exposure descriptors further separate occlusion- and topology-dominant pathways, and small structurally targeted probes cause downstream QA/retrieval degradation comparable to larger-footprint perturbations. These results shift DLA robustness evaluation from footprint-based stress testing toward structure-aware vulnerability auditing.