The Touché23-ValueEval Dataset for Identifying Human Values behind Arguments
This work provides a dataset for researchers in natural language processing and argumentation to identify human values, but it is incremental as it extends an existing dataset.
The authors tackled the problem of automated detection of human values behind arguments by creating the Touché23-ValueEval dataset, which includes 9324 arguments from diverse sources annotated for 54 values, and found that while baseline performance decreased, an out-of-the-box BERT model improved, indicating the dataset enables better model training despite increased classification difficulty.
We present the Touché23-ValueEval Dataset for Identifying Human Values behind Arguments. To investigate approaches for the automated detection of human values behind arguments, we collected 9324 arguments from 6 diverse sources, covering religious texts, political discussions, free-text arguments, newspaper editorials, and online democracy platforms. Each argument was annotated by 3 crowdworkers for 54 values. The Touché23-ValueEval dataset extends the Webis-ArgValues-22. In comparison to the previous dataset, the effectiveness of a 1-Baseline decreases, but that of an out-of-the-box BERT model increases. Therefore, though the classification difficulty increased as per the label distribution, the larger dataset allows for training better models.