ConnPrompt: Connective-cloze Prompt Learning for Implicit Discourse Relation Recognition

Proceedings of the 29th International Conference on Computational Linguistics, pages 902–911 October 12–17, 2022. 902 ConnPrompt: Connective-cloze Prompt Learning for Implicit Discourse Relation Recognition Wei Xiang, Zhenglin Wang, Lu Dai, Bang Wang ∗ School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China {xiangwei, wangzhenglin, dailu18, wangbang}@hust.edu.cn Abstract Implicit Discourse Relation Recognition (IDRR) is to detect and classify relation sense between two text segments without an explicit connective. Vanilla pre-train and fine-tuning paradigm builds upon a Pre-trained Language Model (PLM) with a task-specific neural net- work. However, the task objective functions are often not in accordance with that of the PLM. Furthermore, this paradigm cannot well exploit some linguistic evidence embedded in the pre-training process. The recent pre-train, prompt, and predict paradigm selects appropri- ate prompts to reformulate downstream tasks, so as to utilizing the PLM itself for prediction. However, for its success applications, prompts, verbalizer as well as model training should still be carefully designed for different tasks. As the first trial of using this new paradigm for IDRR, this paper develops a Connective- cloze Prompt (ConnPrompt) to transform the relation prediction task as a connective-cloze task. Specifically, we design two styles of ConnPrompt template: Insert-cloze Prompt (ICP) and Prefix-cloze Prompt (PCP) and construct an answer space mapping to the relation senses based on the hierarchy sense tags and implicit connectives. Furthermore, we use a multi-prompt ensemble to fuse predictions from different prompting results. Experiments on the PDTB corpus show that our method significantly outperforms the state-of-the-art algorithms, even with fewer training data. 1 Introduction Implicit Discourse Relation Recognition (IDRR) aims at detecting and classifying some latent re- lation in between a pair of text segments (called arguments) without an explicit connective word. As illustrated in Fig. 1, an implicit discourse rela- tion of "Contingency" is held between Argument-1 and Argument-2, and the implicit connective ’so’ ∗ Corresponding author: Bang Wang is inserted by annotators in the PDTB corpus. It is of great importance for many downstream Natu- ral Language Processing (NLP) applications, such as question answering (Liakata et al., 2013), ma- chine translation evaluation (Guzmán et al., 2014), information extraction (Xiang and Wang, 2019), sentiment analysis (Wang and Wang, 2020), and etc. However, due to the absence of an explicit con- nective word, inferring discourse relations from the contextual semantics of arguments is still a chal- lenging task. Existing pre-train and fine-tuning paradigm (Liu et al., 2021) builds upon a Pre-trained Language Model (PLM) with a well-designed sophisticated neural network to encode the semantic content and interactive evidence of argument pairs (Liu and Li, 2016; Lei et al., 2017; Bai and Zhao, 2018; Ruan et al., 2020; Li et al., 2020; Liu et al., 2020; Wu et al., 2022). Although the PLMs are adapted to these task-specific neural networks that can ef- fectively learn a kind of contextual semantics of arguments, they introduce some additional param- eters that need to be trained by a large amount of labelled data. Moreover, the task objective function …

Content truncated to 500 words