Using Neighborhood Context to Improve Information Extraction from Visual Documents Captured on Mobile Phones
This improves practical on-device information extraction for mobile users, though it appears incremental as it builds on existing contextual language models.
The paper tackles information extraction from visual documents captured on mobile phones by proposing a Neighborhood-based Information Extraction (NIE) approach that uses local neighborhood context, outperforming state-of-the-art global context-based techniques on two datasets.
Information Extraction from visual documents enables convenient and intelligent assistance to end users. We present a Neighborhood-based Information Extraction (NIE) approach that uses contextual language models and pays attention to the local neighborhood context in the visual documents to improve information extraction accuracy. We collect two different visual document datasets and show that our approach outperforms the state-of-the-art global context-based IE technique. In fact, NIE outperforms existing approaches in both small and large model sizes. Our on-device implementation of NIE on a mobile platform that generally requires small models showcases NIE's usefulness in practical real-world applications.