Improving Document Image Understanding with Reinforcement Finetuning
This addresses the challenge of data scarcity for AI systems in document information extraction, though it is incremental as it builds on existing state-of-the-art extractors.
The paper tackled the problem of improving document image understanding with limited training data by proposing a reinforcement learning-based finetuning method, resulting in consistent performance gains on four datasets, particularly in small-data scenarios.
Successful Artificial Intelligence systems often require numerous labeled data to extract information from document images. In this paper, we investigate the problem of improving the performance of Artificial Intelligence systems in understanding document images, especially in cases where training data is limited. We address the problem by proposing a novel finetuning method using reinforcement learning. Our approach treats the Information Extraction model as a policy network and uses policy gradient training to update the model to maximize combined reward functions that complement the traditional cross-entropy losses. Our experiments on four datasets using labels and expert feedback demonstrate that our finetuning mechanism consistently improves the performance of a state-of-the-art information extractor, especially in the small training data regime.