Transfer Learning for Sequence Labeling Using Source Model and Target Data
This work addresses transfer learning challenges in sequence labeling for domains like Named Entity Recognition, but it is incremental as it builds on existing methods with specific adaptations.
The paper tackles the problem of transferring knowledge from a source neural model for sequence labeling to a target domain with new label categories, without accessing source data, and shows that their approach effectively transfers knowledge and further improves with a neural adapter.
In this paper, we propose an approach for transferring the knowledge of a neural model for sequence labeling, learned from the source domain, to a new model trained on a target domain, where new label categories appear. Our transfer learning (TL) techniques enable to adapt the source model using the target data and new categories, without accessing to the source data. Our solution consists in adding new neurons in the output layer of the target model and transferring parameters from the source model, which are then fine-tuned with the target data. Additionally, we propose a neural adapter to learn the difference between the source and the target label distribution, which provides additional important information to the target model. Our experiments on Named Entity Recognition show that (i) the learned knowledge in the source model can be effectively transferred when the target data contains new categories and (ii) our neural adapter further improves such transfer.