Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents
This addresses the problem of classifying new, unseen document categories in dynamic environments for applications involving semi-structured data, representing an incremental improvement over existing methods.
The paper tackled zero-shot classification of semi-structured documents by proposing a matching-based approach with a pairwise contrastive objective for pretraining and fine-tuning, resulting in a significant boost in Macro F1 scores in both supervised and unsupervised settings.
We investigate semi-structured document classification in a zero-shot setting. Classification of semi-structured documents is more challenging than that of standard unstructured documents, as positional, layout, and style information play a vital role in interpreting such documents. The standard classification setting where categories are fixed during both training and testing falls short in dynamic environments where new document categories could potentially emerge. We focus exclusively on the zero-shot setting where inference is done on new unseen classes. To address this task, we propose a matching-based approach that relies on a pairwise contrastive objective for both pretraining and fine-tuning. Our results show a significant boost in Macro F$_1$ from the proposed pretraining step in both supervised and unsupervised zero-shot settings.