Transfer Learning for Causal Sentence Detection
This work addresses causal relation extraction for text mining, but it is incremental as it shows limited benefits of transfer learning in this context.
The study tackled the problem of detecting causal sentences in text by using transfer learning with ELMO and BERT, finding that it only improves performance on very small datasets, while a baseline method plateaus with larger datasets.
We consider the task of detecting sentences that express causality, as a step towards mining causal relations from texts. To bypass the scarcity of causal instances in relation extraction datasets, we exploit transfer learning, namely ELMO and BERT, using a bidirectional GRU with self-attention (BIGRUATT) as a baseline. We experiment with both generic public relation extraction datasets and a new biomedical causal sentence detection dataset, a subset of which we make publicly available. We find that transfer learning helps only in very small datasets. With larger datasets, BIGRUATT reaches a performance plateau, then larger datasets and transfer learning do not help.