Connective Prediction for Implicit Discourse Relation Recognition via Knowledge Distillation

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5908–5923 July 9-14, 2023 ©2023 Association for Computational Linguistics Connective Prediction for Implicit Discourse Relation Recognition via Knowledge Distillation Hongyi Wu1, Hao Zhou1, Man Lan1,2,3,∗, Yuanbin Wu1 and Yadong Zhang1 1School of Computer Science and Technology, East China Normal University, Shanghai, China 2Shanghai Institute of AI for Education, East China Normal University, Shanghai, China 3Lingang Laboratory, Shanghai, China {hongyiwu,hzhou,yadongzhang}@stu.ecnu.edu.cn {mlan,ybwu}@cs.ecnu.edu.cn Abstract Implicit discourse relation recognition (IDRR) remains a challenging task in discourse anal- ysis due to the absence of connectives. Most existing methods utilize one-hot labels as the sole optimization target, ignoring the internal association among connectives. Besides, these approaches spend lots of effort on template construction, negatively affecting the general- ization capability. To address these problems, we propose a novel Connective Prediction via Knowledge Distillation (CP-KD) approach to instruct large-scale pre-trained language models (PLMs) mining the latent correlations between connectives and discourse relations, which is meaningful for IDRR. Experimen- tal results on the PDTB 2.0/3.0 and CoNLL 2016 datasets show that our method signifi- cantly outperforms the state-of-the-art models on coarse-grained and fine-grained discourse relations. Moreover, our approach can be trans- ferred to explicit discourse relation recognition (EDRR) and achieve acceptable performance. Our code is released in https://github. com/cubenlp/CP_KD-for-IDRR. 1 Introduction Discourse relation recognition (DRR) aims at de- tecting semantic relations between two arguments (sentences or clauses, they are denoted as Arg1 and Arg2, respectively). As illustrated in Figure 1, the discourse relation Contingency (denoted as sense) is held between Arg1 and Arg2, and the explicit connective so is drawn from the raw text while the implicit connective because is manually inserted by annotators. DRR is significant to many natural language processing (NLP) downstream tasks such as causal reasoning (Staliunaite et al., 2021) and question answering (Huang et al., 2021). However, compared with explicit discourse relation recognition (EDRR), implicit discourse relation recognition (IDRR) is still less accurate and practi- cal due to the lack of connectives, which is a major challenge in current discourse analysis research. Public preference is important, product names should match up. Arg1 Arg2 connective (Explicit) It could be a long wait. [[ ] Some investors will tiptoe back in. Arg1 Arg2 connective (Implicit) Sense: Contingency.Cause.Reason Sense: Contingency.Cause.Result Figure 1: Examples of discourse annotation with ex- plicit and implicit connectives in the PDTB 3.0 corpus. The connectives (e.g., because, so, etc.) are crit- ical linguistic cues for identifying discourse rela- tions. On the one hand, with the aid of explicit con- nectives, a simple frequency-based mapping is suf- ficient to achieve over 85% classification accuracy on EDRR (Xue et al., 2016). On the other hand, hu- man annotators utilized connectives to aid relation annotation in the most popular PDTB benchmark datasets (Prasad et al., 2008; Webber et al., 2019). For instance, annotators first manually inserted a connective expression, and then determined the ab- stract relation in consideration of both the implicit connective and argument pairs. Therefore, several studies recognize …

Content truncated to 500 words