Representation Learning for Resource-Constrained Keyphrase Generation
This work addresses the challenge of keyphrase generation for domains with scarce annotated data, offering an incremental improvement over existing methods by leveraging unlabeled data.
The paper tackles the problem of keyphrase generation in domains with limited annotated data by designing a data-oriented approach that uses retrieval-based corpus-level statistics and denoising training objectives to learn task-specific representations from unlabeled documents. The method improves low-resource and zero-shot domain adaptation, especially for absent keyphrases, approaching the performance of models trained with large datasets.
State-of-the-art keyphrase generation methods generally depend on large annotated datasets, limiting their performance in domains with limited annotated data. To overcome this challenge, we design a data-oriented approach that first identifies salient information using retrieval-based corpus-level statistics, and then learns a task-specific intermediate representation based on a pre-trained language model using large-scale unlabeled documents. We introduce salient span recovery and salient span prediction as denoising training objectives that condense the intra-article and inter-article knowledge essential for keyphrase generation. Through experiments on multiple keyphrase generation benchmarks, we show the effectiveness of the proposed approach for facilitating low-resource keyphrase generation and zero-shot domain adaptation. Our method especially benefits the generation of absent keyphrases, approaching the performance of models trained with large training sets.