ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task
This work addresses the gap in evaluating ChatGPT for keyphrase generation, providing insights for researchers and practitioners in natural language processing, though it is incremental as it applies an existing model to a new task.
This study evaluated ChatGPT's performance in keyphrase generation, comparing it to state-of-the-art models across six datasets from scientific and news domains, and found that ChatGPT outperformed all tested models in generating high-quality keyphrases that adapt well to different domains and document lengths.
Transformer-based language models, including ChatGPT, have demonstrated exceptional performance in various natural language generation tasks. However, there has been limited research evaluating ChatGPT's keyphrase generation ability, which involves identifying informative phrases that accurately reflect a document's content. This study seeks to address this gap by comparing ChatGPT's keyphrase generation performance with state-of-the-art models, while also testing its potential as a solution for two significant challenges in the field: domain adaptation and keyphrase generation from long documents. We conducted experiments on six publicly available datasets from scientific articles and news domains, analyzing performance on both short and long documents. Our results show that ChatGPT outperforms current state-of-the-art models in all tested datasets and environments, generating high-quality keyphrases that adapt well to diverse domains and document lengths.