Constrained Multi-Layer Contrastive Learning for Implicit Discourse Relationship Recognition
This work addresses a domain-specific challenge in natural language processing for researchers and practitioners, offering incremental improvements over existing methods.
The paper tackled the problem of implicit discourse relation recognition by proposing a constrained multi-layer contrastive learning method to enhance representation learning, resulting in significant performance improvements on PDTB 2.0 and PDTB 3.0 datasets for both multi-class and binary classification.
Previous approaches to the task of implicit discourse relation recognition (IDRR) generally view it as a classification task. Even with pre-trained language models, like BERT and RoBERTa, IDRR still relies on complicated neural networks with multiple intermediate layers to proper capture the interaction between two discourse units. As a result, the outputs of these intermediate layers may have different capability in discriminating instances of different classes. To this end, we propose to adapt a supervised contrastive learning (CL) method, label- and instance-centered CL, to enhance representation learning. Moreover, we propose a novel constrained multi-layer CL approach to properly impose a constraint that the contrastive loss of higher layers should be smaller than that of lower layers. Experimental results on PDTB 2.0 and PDTB 3.0 show that our approach can significantly improve the performance on both multi-class classification and binary classification.