Towards Reliable and Interpretable Document Question Answering via VLMs
This work addresses the problem of unreliable and non-interpretable answer localization in document question answering for users of VLMs, representing an incremental improvement through a modular approach.
The paper tackles the challenge of inaccurate answer localization in document question answering with Vision-Language Models (VLMs), introducing DocExplainerV0 as a plug-and-play bounding-box prediction module to decouple answer generation from spatial localization. The result is a standardized framework that quantifies the gap between textual accuracy and spatial grounding, establishing a benchmark for future research.
Vision-Language Models (VLMs) have shown strong capabilities in document understanding, particularly in identifying and extracting textual information from complex documents. Despite this, accurately localizing answers within documents remains a major challenge, limiting both interpretability and real-world applicability. To address this, we introduce DocExplainerV0, a plug-and-play bounding-box prediction module that decouples answer generation from spatial localization. This design makes it applicable to existing VLMs, including proprietary systems where fine-tuning is not feasible. Through systematic evaluation, we provide quantitative insights into the gap between textual accuracy and spatial grounding, showing that correct answers often lack reliable localization. Our standardized framework highlights these shortcomings and establishes a benchmark for future research toward more interpretable and robust document information extraction VLMs.