Training LayoutLM from Scratch for Efficient Named-Entity Recognition in the Insurance Domain
This addresses the challenge of limited in-domain data due to privacy constraints for insurance and finance professionals, though it appears incremental as it compares pre-training strategies rather than introducing a fundamentally new approach.
The paper tackles the problem of domain mismatch in pre-trained neural networks for specialized domains like insurance, showing that using domain-relevant documents for pre-training LayoutLM improves named-entity recognition results on a novel dataset of anonymized insurance documents, achieving competitive results with a smaller and faster model.
Generic pre-trained neural networks may struggle to produce good results in specialized domains like finance and insurance. This is due to a domain mismatch between training data and downstream tasks, as in-domain data are often scarce due to privacy constraints. In this work, we compare different pre-training strategies for LayoutLM. We show that using domain-relevant documents improves results on a named-entity recognition (NER) problem using a novel dataset of anonymized insurance-related financial documents called Payslips. Moreover, we show that we can achieve competitive results using a smaller and faster model.