Fine-Grained Evaluation for Implicit Discourse Relation Recognition
This work addresses the challenge of understanding model performance and data scarcity in discourse analysis for NLP researchers, but it is incremental as it builds on existing methods and datasets.
The paper tackles the lack of fine-grained analysis in implicit discourse relation recognition by deeply analyzing pre-trained language model predictions and semi-manually annotating additional data for underrepresented relations in PDTB 3.0, resulting in significant improvement for level-2 senses.
Implicit discourse relation recognition is a challenging task in discourse analysis due to the absence of explicit discourse connectives between spans of text. Recent pre-trained language models have achieved great success on this task. However, there is no fine-grained analysis of the performance of these pre-trained language models for this task. Therefore, the difficulty and possible directions of this task is unclear. In this paper, we deeply analyze the model prediction, attempting to find out the difficulty for the pre-trained language models and the possible directions of this task. In addition to having an in-depth analysis for this task by using pre-trained language models, we semi-manually annotate data to add relatively high-quality data for the relations with few annotated examples in PDTB 3.0. The annotated data significantly help improve implicit discourse relation recognition for level-2 senses.