A Regularization Approach for Incorporating Event Knowledge and Coreference Relations into Neural Discourse Parsing

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2976–2987, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2976 A Regularization Approach for Incorporating Event Knowledge and Coreference Relations into Neural Discourse Parsing Zeyu Dai, Ruihong Huang Department of Computer Science and Engineering Texas A&M University {jzdaizeyu, huangrh}@tamu.edu Abstract We argue that external commonsense knowl- edge and linguistic constraints need to be in- corporated into neural network models for mit- igating data sparsity issues and further improv- ing the performance of discourse parsing. Re- alizing that external knowledge and linguis- tic constraints may not always apply in un- derstanding a particular context, we propose a regularization approach that tightly integrates these constraints with contexts for deriving word representations. Meanwhile, it bal- ances attentions over contexts and constraints through adding a regularization term into the objective function. Experiments show that our knowledge regularization approach out- performs all previous systems on the bench- mark dataset PDTB for discourse parsing. 1 Introduction Discourse parsing and identifying rhetorical dis- course relations between two text spans (i.e., dis- course units, either clauses or sentences) is crucial and beneficial for a wide variety of downstream tasks and applications such as machine transla- tion (Webber et al., 2017), text generation (Mann, 1984; Bosselut et al., 2018) and text summariza- tion (Gerani et al., 2014). In the PDTB-style discourse parsing (Prasad et al., 2008), we commonly distinguish implicit discourse relations from explicit relations, de- pending on whether a discourse connective (e.g., “because”, “however”) appears between two dis- course units. In general, recognizing implicit dis- course relations is more challenging due to the lack of connective, which has recently drawn sig- nificant attention from the NLP researchers. Recent research for implicit discourse rela- tion classification has mostly focused on apply- ing powerful neural network models (Qin et al., 2016a,b; Liu and Li, 2016; Lei et al., 2017; Bai and Zhao, 2018) for modeling compositional meanings and word-level interactions of two dis- course units. More recent research has also ex- ploited utilizing broader contexts (Dai and Huang, 2018) as well as leveraging external training data (Xu et al., 2018). Although progress has been made, the performance of implicit discourse rela- tion identification remains low (macro F1 < 50%). We believe that the low performance is mainly due to the data sparsity issue (Braud and Denis, 2015), which hinders data-thirsty neural network models from making further improvements. Con- sidering the following example from PDTB with two discourse units (DUs): DU1: The editorial of the WHO notes that to- bacco consumption and lung-cancer mortal- ity rates are rising in developing countries. DU2: “No smoking should be established as the norm of social behavior” around the world, the editorial says, through the enactment of laws that limit advertising and promote anti- smoking education. Discourse Relation: Implicit Contingency.Cause Humans can easily recognize this discourse re- lation as “Cause” because we know that “smok- ing” is the key …

Content truncated to 500 words