Universal Language Model Fine-tuning for Text Classification
This addresses the problem of task-specific modifications in NLP for researchers and practitioners, offering a broadly applicable solution with substantial performance gains.
The authors tackled the lack of effective transfer learning in NLP by proposing ULMFiT, a universal fine-tuning method that reduces error by 18-24% on six text classification tasks and matches performance with 100 labeled examples versus 100x more data.
Inductive transfer learning has greatly impacted computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch. We propose Universal Language Model Fine-tuning (ULMFiT), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a language model. Our method significantly outperforms the state-of-the-art on six text classification tasks, reducing the error by 18-24% on the majority of datasets. Furthermore, with only 100 labeled examples, it matches the performance of training from scratch on 100x more data. We open-source our pretrained models and code.