Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6477–6492 July 9-14, 2023 ©2023 Association for Computational Linguistics Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition Haodong Zhao1,2, Ruifang He1,2∗, Mengnan Xiao1,2 and Jing Xu1,2 1College of Intelligence and Computing, Tianjin University, Tianjin, China 2Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin, China {haodongzhao,rfhe,mnxiao,jingxu}@tju.edu.cn Abstract Multi-level implicit discourse relation recogni- tion (MIDRR) aims at identifying hierarchical discourse relations among arguments. Previous methods achieve the promotion through fine- tuning PLMs. However, due to the data scarcity and the task gap, the pre-trained feature space cannot be accurately tuned to the task-specific space, which even aggravates the collapse of the vanilla space. Besides, the comprehension of hierarchical semantics for MIDRR makes the conversion much harder. In this paper, we pro- pose a prompt-based Parameter-Efficient Multi- level IDRR (PEMI) framework to solve the above problems. First, we leverage parameter- efficient prompt tuning to drive the inputted arguments to match the pre-trained space and realize the approximation with few parameters. Furthermore, we propose a hierarchical label re- fining (HLR) method for the prompt verbalizer to deeply integrate hierarchical guidance into the prompt tuning. Finally, our model achieves comparable results on PDTB 2.0 and 3.0 us- ing about 0.1% trainable parameters compared with baselines and the visualization demon- strates the effectiveness of our HLR method. 1 Introduction Implicit discourse relation recognition (IDRR) (Pitler et al., 2009) is one of the most vital sub-tasks in discourse analysis, which proposes to discover the discourse relation between two discourse argu- ments without the guidance of explicit connectives. Due to the lack of connectives, the model can only recognize the relations through semantic clues and entity anaphora between arguments, which makes IDRR a challenging task. Through a deeper un- derstanding of this task, it is beneficial to a series of downstream tasks such as text summarization (Li et al., 2020b), dialogue summarization (Feng et al., 2021) and event relation extraction (Tang et al., 2021). Meanwhile, the discourse relation is ∗The Corresponding author. Arg1: After the race, Fortune 500 executives drooled like schoolboys over the cars and drivers. Arg2: No dummies, the drivers pointed out they still had space on their machines for another sponsor's name or two. Root Level 1: Contingency Level 2: Cause Level 3: Result Connective: Consequently ... ... Figure 1: An Instance for multi-level IDRR. annotated as multi-level labels. As shown in Fig- ure 1, the top-level label of this argument pair is Comparison, while the sub-label Contrast is the fine-grained semantic expression of Comparison. Beyond that, when annotating the implicit relation, the annotators simulate adding a connective Conse- quently. We regard these connectives as the bottom level of discourse relations. Since pre-trained language models (PLMs) are widely applied, IDRR has also achieved consider- able improvement. However, previous work (Xu et al., 2018; Shi et al., 2018; Dou et al., 2021) has mentioned the data scarcity of the IDRR, in …

Content truncated to 500 words