Interactively-Propagative Attention Learning for Implicit Discourse Relation Recognition

Proceedings of the 28th International Conference on Computational Linguistics, pages 3168–3178 Barcelona, Spain (Online), December 8-13, 2020 3168 Interactively-Propagative Attention Learning for Implicit Discourse Relation Recognition Huibin Ruan† Yu Hong†∗ Yang Xu† Zhen Huang‡ Guodong Zhou† Min Zhang† † School of Computer Science and Technology, Soochow University, 1 Shizi, Suzhou, CHN ‡ School of Computer Science, National University of Defense Technology, Changsha, CHN †{hbr416, tianxianer, andreaxu41}@gmail.com; ‡huangzhen@nudt.edu.cn; †{gdzhou, mzhang}@suda.edu.cn Abstract We tackle implicit discourse relation recognition. Both self-attention and interactive-attention mechanisms have been applied for attention-aware representation learning, which improves the current discourse analysis models. To take advantages of the two attention mechanisms simulta- neously, we develop a propagative attention learning model using a cross-coupled two-channel network. We experiment on Penn Discourse Treebank. The test results demonstrate that our model yields substantial improvements over the baselines (BiLSTM and BERT). 1 Introduction Implicit Discourse Relation Recognition (IDRR) is required to determine the relationship between ar- guments, under the condition that there is lack of a connective signaling the relationship. An argument generally stands for a narrative sentence or clause. For example, the arguments (i.e., Arg1 and Arg2) in Figure 1 hold a causal relation, where the possible connective “because” is not given. [Arg1: Psyllium’s not a good crop.] [Arg2: You get a rain at the wrong time and the crop is ruined.] Self-attention Interactive attention Figure 1: An example of causally-related arguments. Since the time when IDRR was boiled down to a problem of discourse relation classification (Pitler et al., 2009; Lin et al., 2014), intense interest has been devoted to the study of argument representation and neural relation classification (Zhang et al., 2015; Liu et al., 2016; Chen et al., 2016; Lan et al., 2017; Bai and Zhao, 2018; Nguyen et al., 2019). Context-specific non-interactive attention mechanism (also referred to self-attention mechanism) (Lin et al., 2017) and companion-dependent interactive attention mechanism (Ma et al., 2017; Meng et al., 2016) have been used for enhancing the sentence-level embed- ding process. Both are proven effective in argument encoding (Guo et al., 2018; Liu and Li, 2016) as well as the perception of discourse relations. During encoding, the self-attention mechanism is able to highlight the latent information of attention-worthy words conditioned on local context (Note: we refer the attention-worthy words to the ones which play the dominant role in signaling discourse relations). By contrast, the interactive attention mechanism introduces external evidence into the identification of attention-worthy words, and similarly, highlighting their latent information. So far, the two kinds of attention computations are performed separately. However, our survey shows that, in some cases, the context-specific self-attentive information can be inherited by the interaction- based attention computation. Let’s consider the words “rain at the wrong time” in Arg2 in Figure 1. On the one hand, those words may catch the attention of the self-attention mechanism due to the occurrence ∗ Corresponding author: Yu Hong (tianxianer@gmail.com) This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/. 3169 Fully- Connected Layer …

Content truncated to 500 words