Encoding and Fusing Semantic Connection and Linguistic Evidence for Implicit Discourse Relation Recognition

Findings of the Association for Computational Linguistics: ACL 2022, pages 3247 - 3257 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Encoding and Fusing Semantic Connection and Linguistic Evidence for Implicit Discourse Relation Recognition Wei Xiang1, Bang Wang1, Lu Dai1, Yijun Mo2∗ 1School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, China 2School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China {xiangwei, wangbang, dailu18, moyj}@hust.edu.cn Abstract Prior studies use one attention mechanism to improve contextual semantic representa- tion learning for implicit discourse relation recognition (IDRR). However, diverse relation senses may benefit from different attention mechanisms. We also argue that some linguis- tic relation in between two words can be fur- ther exploited for IDRR. This paper proposes a Multi-Attentive Neural Fusion (MANF) model to encode and fuse both semantic connection and linguistic evidence for IDRR. In MANF, we design a Dual Attention Network (DAN) to learn and fuse two kinds of attentive rep- resentation for arguments as its semantic con- nection. We also propose an Offset Matrix Network (OMN) to encode the linguistic rela- tions of word-pairs as linguistic evidence. Our MANF model achieves the state-of-the-art re- sults on the PDTB 3.0 corpus. 1 Introduction Implicit Discourse Relation Recognition (IDRR) is to detect and classify some latent relation in be- tween a pair of text segments (called arguments) without an explicit connective word. It is of great importance for many downstream Natural Lan- guage Processing (NLP) applications, such as ques- tion answering (Liakata et al., 2013), machine trans- lation (Guzmán et al., 2014), information extrac- tion (Xiang and Wang, 2019), sentiment analy- sis (Wang and Wang, 2020), and etc. However, due to the absence of an explicit connective word, inferring discourse relations from the contextual semantics of arguments is still a challenging task. Conventional machine learning based methods usually train a relation classifier by using many handmade features to capture lexical, syntactic reg- ularity and contextual information of arguments, which is time-consuming and labor-intensive (Pitler et al., 2009, 2008). Deep learning based methods ∗ Corresponding author: Yijun Mo design diverse neural networks to automatic learn the contextual semantic representation of each ar- gument, such as the Shallow Conventional Neural Network (SCNN) (Zhang et al., 2015), Tree-like Long Short-Term Memory (Tree-LSTM) (Ruther- ford et al., 2017), and BiLSTM-CNN frame- work (Guo et al., 2019). Although these neural networks can autonomously learn a kind of deeper contextual semantics of arguments, they do not dif- ferentiate arguments’ words in the representation learning. Recently, some attention mechanisms have been employed in neural networks to unequally treat words in representation learning. For example, the self-attention computes the local contextual impor- tance of each word in one argument, which gen- erally prioritizes content words for better learning substantive meaning of an argument (Zhou et al., 2016). The interactive attention weights each word in one argument according to its interaction with the representation of another argument, which usu- ally focuses on the rhetorical device of two argu- ments, …

Content truncated to 500 words