Measuring Your ASTE Models in The Wild: A Diversified Multi-domain Dataset For Aspect Sentiment Triplet Extraction
This provides a more realistic benchmark for ASTE research, addressing limitations in current datasets, though it is incremental as it focuses on data creation rather than method innovation.
The authors tackled the lack of real-world diversity in Aspect Sentiment Triplet Extraction (ASTE) datasets by introducing DMASTE, a manually annotated dataset with varied lengths, expressions, aspect types, and domains, which they show is more challenging than existing datasets.
Aspect Sentiment Triplet Extraction (ASTE) is widely used in various applications. However, existing ASTE datasets are limited in their ability to represent real-world scenarios, hindering the advancement of research in this area. In this paper, we introduce a new dataset, named DMASTE, which is manually annotated to better fit real-world scenarios by providing more diverse and realistic reviews for the task. The dataset includes various lengths, diverse expressions, more aspect types, and more domains than existing datasets. We conduct extensive experiments on DMASTE in multiple settings to evaluate previous ASTE approaches. Empirical results demonstrate that DMASTE is a more challenging ASTE dataset. Further analyses of in-domain and cross-domain settings provide promising directions for future research. Our code and dataset are available at https://github.com/NJUNLP/DMASTE.