Self-Supervised Pre-Training for Table Structure Recognition Transformer
This work addresses a specific bottleneck in table structure recognition for researchers and practitioners, offering an incremental improvement over existing methods.
The paper tackles the performance drop when replacing CNN backbones with linear projection transformers in table structure recognition by introducing a self-supervised pre-training method, which mitigates the gap and achieves competitive results on benchmark datasets.
Table structure recognition (TSR) aims to convert tabular images into a machine-readable format. Although hybrid convolutional neural network (CNN)-transformer architecture is widely used in existing approaches, linear projection transformer has outperformed the hybrid architecture in numerous vision tasks due to its simplicity and efficiency. However, existing research has demonstrated that a direct replacement of CNN backbone with linear projection leads to a marked performance drop. In this work, we resolve the issue by proposing a self-supervised pre-training (SSP) method for TSR transformers. We discover that the performance gap between the linear projection transformer and the hybrid CNN-transformer can be mitigated by SSP of the visual encoder in the TSR model. We conducted reproducible ablation studies and open-sourced our code at https://github.com/poloclub/unitable to enhance transparency, inspire innovations, and facilitate fair comparisons in our domain as tables are a promising modality for representation learning.