The Limits of ChatGPT in Extracting Aspect-Category-Opinion-Sentiment Quadruples: A Comparative Analysis
This work addresses a complex task in aspect-based sentiment analysis for researchers and practitioners, but it is incremental as it adapts existing methods to ChatGPT.
The paper tackled the problem of extracting aspect-category-opinion-sentiment quadruples from texts using ChatGPT, developing a specialized prompt template and few-shot selection method, and found that ChatGPT performs competitively but has limitations compared to state-of-the-art models on four public datasets.
Recently, ChatGPT has attracted great attention from both industry and academia due to its surprising abilities in natural language understanding and generation. We are particularly curious about whether it can achieve promising performance on one of the most complex tasks in aspect-based sentiment analysis, i.e., extracting aspect-category-opinion-sentiment quadruples from texts. To this end, in this paper we develop a specialized prompt template that enables ChatGPT to effectively tackle this complex quadruple extraction task. Further, we propose a selection method on few-shot examples to fully exploit the in-context learning ability of ChatGPT and uplift its effectiveness on this complex task. Finally, we provide a comparative evaluation on ChatGPT against existing state-of-the-art quadruple extraction models based on four public datasets and highlight some important findings regarding the capability boundaries of ChatGPT in the quadruple extraction.