Argumentative Large Language Models for Explainable and Contestable Claim Verification
This addresses the limitation of LLMs in providing explainable and contestable outputs for decision-making tasks, offering a novel approach to enhance trust and correctness in AI systems, though it is incremental in combining existing argumentation techniques with LLMs.
The paper tackles the problem of making large language models (LLMs) more explainable and contestable for decision-making, specifically in claim verification, by introducing argumentative LLMs (ArgLLMs) that use argumentation frameworks for formal reasoning, resulting in improved interpretability and competitive performance compared to state-of-the-art methods.
The profusion of knowledge encoded in large language models (LLMs) and their ability to apply this knowledge zero-shot in a range of settings makes them promising candidates for use in decision-making. However, they are currently limited by their inability to provide outputs which can be faithfully explained and effectively contested to correct mistakes. In this paper, we attempt to reconcile these strengths and weaknesses by introducing \emph{argumentative LLMs (ArgLLMs)}, a method for augmenting LLMs with argumentative reasoning. Concretely, ArgLLMs construct argumentation frameworks, which then serve as the basis for formal reasoning in support of decision-making. The interpretable nature of these argumentation frameworks and formal reasoning means that any decision made by ArgLLMs may be explained and contested. We evaluate ArgLLMs' performance experimentally in comparison with state-of-the-art techniques, in the context of the decision-making task of claim verification. We also define novel properties to characterise contestability and assess ArgLLMs formally in terms of these properties.