Cross-Domain Robustness of Transformer-based Keyphrase Generation
This addresses the issue of domain adaptation for keyphrase generation in NLP, which is incremental as it builds on existing models and tasks.
The paper tackled the problem of cross-domain robustness in keyphrase generation using abstractive text summarization models, finding that fine-tuned BART models show high performance on target corpora but low zero-shot results on others, with preliminary out-of-domain fine-tuning improving performance on small datasets.
Modern models for text generation show state-of-the-art results in many natural language processing tasks. In this work, we explore the effectiveness of abstractive text summarization models for keyphrase selection. A list of keyphrases is an important element of a text in databases and repositories of electronic documents. In our experiments, abstractive text summarization models fine-tuned for keyphrase generation show quite high results for a target text corpus. However, in most cases, the zero-shot performance on other corpora and domains is significantly lower. We investigate cross-domain limitations of abstractive text summarization models for keyphrase generation. We present an evaluation of the fine-tuned BART models for the keyphrase selection task across six benchmark corpora for keyphrase extraction including scientific texts from two domains and news texts. We explore the role of transfer learning between different domains to improve the BART model performance on small text corpora. Our experiments show that preliminary fine-tuning on out-of-domain corpora can be effective under conditions of a limited number of samples.