Dynamic Knowledge Integration for Evidence-Driven Counter-Argument Generation with Large Language Models
This addresses the need for more controlled and factual counter-argument generation systems, though it is incremental as it builds on existing LLM and retrieval methods.
The paper tackles the problem of generating evidence-based counter-arguments with LLMs by integrating dynamic external knowledge from the web, resulting in significant improvements in relatedness, persuasiveness, and factuality.
This paper investigates the role of dynamic external knowledge integration in improving counter-argument generation using Large Language Models (LLMs). While LLMs have shown promise in argumentative tasks, their tendency to generate lengthy, potentially unfactual responses highlights the need for more controlled and evidence-based approaches. We introduce a new manually curated dataset of argument and counter-argument pairs specifically designed to balance argumentative complexity with evaluative feasibility. We also propose a new LLM-as-a-Judge evaluation methodology that shows a stronger correlation with human judgments compared to traditional reference-based metrics. Our experimental results demonstrate that integrating dynamic external knowledge from the web significantly improves the quality of generated counter-arguments, particularly in terms of relatedness, persuasiveness, and factuality. The findings suggest that combining LLMs with real-time external knowledge retrieval offers a promising direction for developing more effective and reliable counter-argumentation systems.