RoDLA: Benchmarking the Robustness of Document Layout Analysis Models
This addresses the need for robust DLA models in real-world document processing applications, representing a novel benchmark but incremental method improvements.
The paper tackles the underexplored problem of robustness in Document Layout Analysis (DLA) models by introducing a benchmark with 450K images and 36 perturbations, resulting in the RoDLA model achieving state-of-the-art mRD scores of 115.7, 135.4, and 150.4 and mAP improvements of +3.8%, +7.1%, and +12.1%.
Before developing a Document Layout Analysis (DLA) model in real-world applications, conducting comprehensive robustness testing is essential. However, the robustness of DLA models remains underexplored in the literature. To address this, we are the first to introduce a robustness benchmark for DLA models, which includes 450K document images of three datasets. To cover realistic corruptions, we propose a perturbation taxonomy with 36 common document perturbations inspired by real-world document processing. Additionally, to better understand document perturbation impacts, we propose two metrics, Mean Perturbation Effect (mPE) for perturbation assessment and Mean Robustness Degradation (mRD) for robustness evaluation. Furthermore, we introduce a self-titled model, i.e., Robust Document Layout Analyzer (RoDLA), which improves attention mechanisms to boost extraction of robust features. Experiments on the proposed benchmarks (PubLayNet-P, DocLayNet-P, and M$^6$Doc-P) demonstrate that RoDLA obtains state-of-the-art mRD scores of 115.7, 135.4, and 150.4, respectively. Compared to previous methods, RoDLA achieves notable improvements in mAP of +3.8%, +7.1% and +12.1%, respectively.