Redefining Absent Keyphrases and their Effect on Retrieval Effectiveness
This work addresses the need for better evaluation of keyphrase generation models in information retrieval, offering incremental improvements in understanding their impact.
The paper tackles the problem of evaluating absent keyphrases in neural keyphrase generation by introducing a finer-grained categorization scheme, revealing that only about 20% of words in these keyphrases contribute to document expansion but drive significant gains in retrieval effectiveness.
Neural keyphrase generation models have recently attracted much interest due to their ability to output absent keyphrases, that is, keyphrases that do not appear in the source text. In this paper, we discuss the usefulness of absent keyphrases from an Information Retrieval (IR) perspective, and show that the commonly drawn distinction between present and absent keyphrases is not made explicit enough. We introduce a finer-grained categorization scheme that sheds more light on the impact of absent keyphrases on scientific document retrieval. Under this scheme, we find that only a fraction (around 20%) of the words that make up keyphrases actually serves as document expansion, but that this small fraction of words is behind much of the gains observed in retrieval effectiveness. We also discuss how the proposed scheme can offer a new angle to evaluate the output of neural keyphrase generation models.