LAD-RAG: Layout-aware Dynamic RAG for Visually-Rich Document Understanding
This work solves the problem of incomplete evidence retrieval for multi-page reasoning tasks in visually rich document understanding, offering a domain-specific improvement.
The paper tackled the problem of question answering over visually rich documents by addressing limitations in conventional RAG methods that lose structural and cross-page dependencies, proposing LAD-RAG, which improved retrieval with over 90% perfect recall on average and outperformed baselines by up to 20% in recall, leading to higher QA accuracy.
Question answering over visually rich documents (VRDs) requires reasoning not only over isolated content but also over documents' structural organization and cross-page dependencies. However, conventional retrieval-augmented generation (RAG) methods encode content in isolated chunks during ingestion, losing structural and cross-page dependencies, and retrieve a fixed number of pages at inference, regardless of the specific demands of the question or context. This often results in incomplete evidence retrieval and degraded answer quality for multi-page reasoning tasks. To address these limitations, we propose LAD-RAG, a novel Layout-Aware Dynamic RAG framework. During ingestion, LAD-RAG constructs a symbolic document graph that captures layout structure and cross-page dependencies, adding it alongside standard neural embeddings to yield a more holistic representation of the document. During inference, an LLM agent dynamically interacts with the neural and symbolic indices to adaptively retrieve the necessary evidence based on the query. Experiments on MMLongBench-Doc, LongDocURL, DUDE, and MP-DocVQA demonstrate that LAD-RAG improves retrieval, achieving over 90% perfect recall on average without any top-k tuning, and outperforming baseline retrievers by up to 20% in recall at comparable noise levels, yielding higher QA accuracy with minimal latency.