Information Extraction from Documents: Question Answering vs Token Classification in real-world setups
This work provides practical guidance for researchers and practitioners in document intelligence on when to use QA vs. token classification methods, though it is incremental as it benchmarks existing approaches.
The paper compared question answering (QA) and token classification approaches for document key information extraction, finding that token classification performs better on clean, short entities, while QA is more effective in noisy environments or for extracting long entities.
Research in Document Intelligence and especially in Document Key Information Extraction (DocKIE) has been mainly solved as Token Classification problem. Recent breakthroughs in both natural language processing (NLP) and computer vision helped building document-focused pre-training methods, leveraging a multimodal understanding of the document text, layout and image modalities. However, these breakthroughs also led to the emergence of a new DocKIE subtask of extractive document Question Answering (DocQA), as part of the Machine Reading Comprehension (MRC) research field. In this work, we compare the Question Answering approach with the classical token classification approach for document key information extraction. We designed experiments to benchmark five different experimental setups : raw performances, robustness to noisy environment, capacity to extract long entities, fine-tuning speed on Few-Shot Learning and finally Zero-Shot Learning. Our research showed that when dealing with clean and relatively short entities, it is still best to use token classification-based approach, while the QA approach could be a good alternative for noisy environment or long entities use-cases.