Contextualizing Argument Quality Assessment with Relevant Knowledge
This addresses the challenge of misinformation and targeted speech by improving the accuracy and generalizability of argument quality assessment, though it is incremental as it builds on existing computational methods.
The paper tackles the problem of automatic argument quality assessment by introducing SPARK, a method that contextualizes arguments with relevant knowledge, resulting in consistent outperformance of existing techniques across multiple metrics in both in-domain and zero-shot setups.
Automatic assessment of the quality of arguments has been recognized as a challenging task with significant implications for misinformation and targeted speech. While real-world arguments are tightly anchored in context, existing computational methods analyze their quality in isolation, which affects their accuracy and generalizability. We propose SPARK: a novel method for scoring argument quality based on contextualization via relevant knowledge. We devise four augmentations that leverage large language models to provide feedback, infer hidden assumptions, supply a similar-quality argument, or give a counter-argument. SPARK uses a dual-encoder Transformer architecture to enable the original argument and its augmentation to be considered jointly. Our experiments in both in-domain and zero-shot setups show that SPARK consistently outperforms existing techniques across multiple metrics.