An Exploration of Data Efficiency in Intra-Dataset Task Transfer for Dialog Understanding
This work addresses data efficiency for NLP researchers, but the results are incremental as they highlight limitations rather than improvements.
The study investigated how varying amounts of target task training data affect sequential transfer learning in dialog understanding, finding that data size often has minimal impact on performance compared to models without transfer learning, suggesting issues like catastrophic forgetting.
Transfer learning is an exciting area of Natural Language Processing that has the potential to both improve model performance and increase data efficiency. This study explores the effects of varying quantities of target task training data on sequential transfer learning in the dialog domain. We hypothesize that a model can utilize the information learned from a source task to better learn a target task, thereby reducing the number of target task training samples required. Unintuitively, our data shows that often target task training data size has minimal effect on how sequential transfer learning performs compared to the same model without transfer learning. Our results lead us to believe that this unexpected result could be due to the effects of catastrophic forgetting, motivating further work into methods that prevent such forgetting.