QANA: LLM-based Question Generation and Network Analysis for Zero-shot Key Point Analysis and Beyond
This addresses information overload in social media analysis by providing a flexible, zero-shot method for key point extraction, though it is incremental as it builds on existing LLM and graph techniques.
The paper tackles opinion mining by proposing QANA, a framework that uses LLMs to generate questions from comments and constructs bipartite graphs to analyze opinion importance via centrality measures, achieving comparable performance to supervised models in zero-shot key point matching with linear computational cost.
The proliferation of social media has led to information overload and increased interest in opinion mining. We propose "Question-Answering Network Analysis" (QANA), a novel opinion mining framework that utilizes Large Language Models (LLMs) to generate questions from users' comments, constructs a bipartite graph based on the comments' answerability to the questions, and applies centrality measures to examine the importance of opinions. We investigate the impact of question generation styles, LLM selections, and the choice of embedding model on the quality of the constructed QA networks by comparing them with annotated Key Point Analysis datasets. QANA achieves comparable performance to previous state-of-the-art supervised models in a zero-shot manner for Key Point Matching task, also reducing the computational cost from quadratic to linear. For Key Point Generation, questions with high PageRank or degree centrality align well with manually annotated key points. Notably, QANA enables analysts to assess the importance of key points from various aspects according to their selection of centrality measure. QANA's primary contribution lies in its flexibility to extract key points from a wide range of perspectives, which enhances the quality and impartiality of opinion mining.