Improving NER's Performance with Massive financial corpus
This work addresses the challenge of high annotation costs for small businesses in financial NER, though it appears incremental in its approach.
The paper tackled the problem of training deep neural networks for company-name recognition with limited high-quality annotation data by using pre-trained lite language models, knowledge distillation, and multi-stage learning, achieving a recall rate increase of nearly 20 points and a 4x speedup compared to a BERT-CRF model.
Training large deep neural networks needs massive high quality annotation data, but the time and labor costs are too expensive for small business. We start a company-name recognition task with a small scale and low quality training data, then using skills to enhanced model training speed and predicting performance with minimum labor cost. The methods we use involve pre-training a lite language model such as Albert-small or Electra-small in financial corpus, knowledge of distillation and multi-stage learning. The result is that we raised the recall rate by nearly 20 points and get 4 times as fast as BERT-CRF model.