Prompted Aspect Key Point Analysis for Quantitative Review Summarization
This work addresses the issue of inaccurate quantification in review summarization for businesses, offering a more faithful approach without requiring large annotated datasets.
The paper tackles the problem of generating overlapping and hallucinated key points in quantitative review summarization by proposing Prompted Aspect Key Point Analysis (PAKPA), which uses aspect sentiment analysis and prompted in-context learning with LLMs to achieve state-of-the-art performance on Yelp and SPACE datasets.
Key Point Analysis (KPA) aims for quantitative summarization that provides key points (KPs) as succinct textual summaries and quantities measuring their prevalence. KPA studies for arguments and reviews have been reported in the literature. A majority of KPA studies for reviews adopt supervised learning to extract short sentences as KPs before matching KPs to review comments for quantification of KP prevalence. Recent abstractive approaches still generate KPs based on sentences, often leading to KPs with overlapping and hallucinated opinions, and inaccurate quantification. In this paper, we propose Prompted Aspect Key Point Analysis (PAKPA) for quantitative review summarization. PAKPA employs aspect sentiment analysis and prompted in-context learning with Large Language Models (LLMs) to generate and quantify KPs grounded in aspects for business entities, which achieves faithful KPs with accurate quantification, and removes the need for large amounts of annotated data for supervised training. Experiments on the popular review dataset Yelp and the aspect-oriented review summarization dataset SPACE show that our framework achieves state-of-the-art performance. Source code and data are available at: https://github.com/antangrocket1312/PAKPA