CTE: A Dataset for Contextualized Table Extraction
This dataset addresses the problem of fragmented benchmarks for researchers in document AI, though it is incremental as it builds on existing datasets like PubTables-1M and PubLayNet.
The authors tackled the lack of a unified dataset for document layout analysis and table understanding by creating CTE, a dataset of 75k annotated pages from scientific papers with over 35k tables, enabling tasks like table detection and structure recognition.
Relevant information in documents is often summarized in tables, helping the reader to identify useful facts. Most benchmark datasets support either document layout analysis or table understanding, but lack in providing data to apply both tasks in a unified way. We define the task of Contextualized Table Extraction (CTE), which aims to extract and define the structure of tables considering the textual context of the document. The dataset comprises 75k fully annotated pages of scientific papers, including more than 35k tables. Data are gathered from PubMed Central, merging the information provided by annotations in the PubTables-1M and PubLayNet datasets. The dataset can support CTE and adds new classes to the original ones. The generated annotations can be used to develop end-to-end pipelines for various tasks, including document layout analysis, table detection, structure recognition, and functional analysis. We formally define CTE and evaluation metrics, showing which subtasks can be tackled, describing advantages, limitations, and future works of this collection of data. Annotations and code will be accessible a https://github.com/AILab-UniFI/cte-dataset.