Can Large Language Models perform Relation-based Argument Mining?
This work addresses the challenge of automating argument analysis for online debates, offering a potential improvement over existing methods, though it is incremental as it applies existing LLMs to a specific task.
The paper tackled the problem of relation-based argument mining (RbAM), which identifies support and disagreement relations between arguments, by showing that appropriately primed and prompted large language models (LLMs) like Llama-2 and Mistral significantly outperform the best RoBERTa-based baseline across ten datasets.
Argument mining (AM) is the process of automatically extracting arguments, their components and/or relations amongst arguments and components from text. As the number of platforms supporting online debate increases, the need for AM becomes ever more urgent, especially in support of downstream tasks. Relation-based AM (RbAM) is a form of AM focusing on identifying agreement (support) and disagreement (attack) relations amongst arguments. RbAM is a challenging classification task, with existing methods failing to perform satisfactorily. In this paper, we show that general-purpose Large Language Models (LLMs), appropriately primed and prompted, can significantly outperform the best performing (RoBERTa-based) baseline. Specifically, we experiment with two open-source LLMs (Llama-2 and Mistral) with ten datasets.