A Better Choice: Entire-space Datasets for Aspect Sentiment Triplet Extraction
This work solves a reproducibility and evaluation problem for researchers in natural language processing by standardizing datasets to better reflect real-world scenarios.
The paper addresses the inconsistency in dataset versions used for Aspect Sentiment Triplet Extraction (ASTE) by proposing the use of entire-space datasets, which include sentences without triplets and aspect terms not in triplets, leading to more realistic evaluations and improved model performance.
Aspect sentiment triplet extraction (ASTE) aims to extract aspect term, sentiment and opinion term triplets from sentences. Since the initial datasets used to evaluate models on ASTE had flaws, several studies later corrected the initial datasets and released new versions of the datasets independently. As a result, different studies select different versions of datasets to evaluate their methods, which makes ASTE-related works hard to follow. In this paper, we analyze the relation between different versions of datasets and suggest that the entire-space version should be used for ASTE. Besides the sentences containing triplets and the triplets in the sentences, the entire-space version additionally includes the sentences without triplets and the aspect terms which do not belong to any triplets. Hence, the entire-space version is consistent with real-world scenarios and evaluating models on the entire-space version can better reflect the models' performance in real-world scenarios. In addition, experimental results show that evaluating models on non-entire-space datasets inflates the performance of existing models and models trained on the entire-space version can obtain better performance.