Keyphrase Generation Beyond the Boundaries of Title and Abstract
This addresses the limitation of relying only on titles and abstracts for keyphrase generation in scholarly domains, offering a dataset and method that could enhance document indexing and retrieval, though it is incremental as it builds on existing neural models.
The paper tackles the problem of keyphrase generation in scholarly articles by exploring whether using full text or similar articles improves performance, finding that adding sentences from the full text, especially as an extractive summary, significantly boosts generation of both present and absent keyphrases.
Keyphrase generation aims at generating important phrases (keyphrases) that best describe a given document. In scholarly domains, current approaches have largely used only the title and abstract of the articles to generate keyphrases. In this paper, we comprehensively explore whether the integration of additional information from the full text of a given article or from semantically similar articles can be helpful for a neural keyphrase generation model or not. We discover that adding sentences from the full text, particularly in the form of the extractive summary of the article can significantly improve the generation of both types of keyphrases that are either present or absent from the text. Experimental results with three widely used models for keyphrase generation along with one of the latest transformer models suitable for longer documents, Longformer Encoder-Decoder (LED) validate the observation. We also present a new large-scale scholarly dataset FullTextKP for keyphrase generation. Unlike prior large-scale datasets, FullTextKP includes the full text of the articles along with the title and abstract. We release the source code at https://github.com/kgarg8/FullTextKP.