FinBERT: A Pretrained Language Model for Financial Communications
This provides a specialized tool for practitioners and researchers in financial NLP, though it is incremental as it adapts an existing method to a new domain.
The authors tackled the lack of pretrained language models for financial communications by developing FinBERT, a domain-specific BERT model trained on large-scale financial text corpora, which outperformed generic BERT in three financial sentiment classification tasks.
Contextual pretrained language models, such as BERT (Devlin et al., 2019), have made significant breakthrough in various NLP tasks by training on large scale of unlabeled text re-sources.Financial sector also accumulates large amount of financial communication text.However, there is no pretrained finance specific language models available. In this work,we address the need by pretraining a financial domain specific BERT models, FinBERT, using a large scale of financial communication corpora. Experiments on three financial sentiment classification tasks confirm the advantage of FinBERT over generic domain BERT model. The code and pretrained models are available at https://github.com/yya518/FinBERT. We hope this will be useful for practitioners and researchers working on financial NLP tasks.