UniTabNet: Bridging Vision and Language Models for Enhanced Table Structure Recognition
This work addresses the challenge of accurately parsing table structures for processing tabular data, which is incremental as it builds on existing methods by incorporating textual semantics.
The paper tackles the problem of table structure recognition by addressing the limitation of previous methods in comprehending textual semantics, particularly in descriptive cells, and introduces UniTabNet, a framework that integrates vision and language models to achieve state-of-the-art performance on datasets like PubTabNet and PubTables1M.
In the digital era, table structure recognition technology is a critical tool for processing and analyzing large volumes of tabular data. Previous methods primarily focus on visual aspects of table structure recovery but often fail to effectively comprehend the textual semantics within tables, particularly for descriptive textual cells. In this paper, we introduce UniTabNet, a novel framework for table structure parsing based on the image-to-text model. UniTabNet employs a ``divide-and-conquer'' strategy, utilizing an image-to-text model to decouple table cells and integrating both physical and logical decoders to reconstruct the complete table structure. We further enhance our framework with the Vision Guider, which directs the model's focus towards pertinent areas, thereby boosting prediction accuracy. Additionally, we introduce the Language Guider to refine the model's capability to understand textual semantics in table images. Evaluated on prominent table structure datasets such as PubTabNet, PubTables1M, WTW, and iFLYTAB, UniTabNet achieves a new state-of-the-art performance, demonstrating the efficacy of our approach. The code will also be made publicly available.