Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1299–1308 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification Man Lan1,2, Jianxiang Wang1,3∗, Yuanbin Wu1,2∗, Zheng-Yu Niu3∗ , Haifeng Wang3 1 School of Computer Science and Software Engineering, East China Normal University 2 Shanghai Key Laboratory of Multidimensional Information Processing, P.R.China 3 Baidu Inc., Beijing, P.R.China {mlan,ybwu}@cs.ecnu.edu.cn {wangjianxiang01,niuzhengyu,wanghaifeng}@baidu.com Abstract We present a novel multi-task attention- based neural network model to address implicit discourse relationship representa- tion and identification through two type- s of representation learning, an attention- based neural network for learning dis- course relationship representation with t- wo arguments and a multi-task framework for learning knowledge from annotated and unannotated corpora. The extensive experiments have been performed on t- wo benchmark corpora (i.e., PDTB and CoNLL-2016 datasets). Experimental re- sults show that our proposed model out- performs the state-of-the-art systems on benchmark corpora. 1 Introduction The task of implicit discourse relation (or rhetor- ical relation) identification is to recognize how t- wo adjacent text spans without explicit discourse marker (i.e., connective, e.g., because or but ) be- tween them are logically connected to one anoth- er (e.g., cause or contrast). It is considered to be a crucial step for discourse analysis and lan- guage generation and helpful to many downstream NLP applications, e.g., QA, MT, sentiment analy- sis, machine comprehension, etc. With the release of PDTB 2.0 (Prasad et al., 2008), lots of work has been done for discourse re- lation identification on natural (i.e., genuine) dis- course data (Pitler et al., 2009; Lin et al., 2009; Wang et al., 2010; Zhou et al., 2010; Braud and Denis, 2015; Fisher and Simmons, 2015) with the use of traditional NLP linguistically informed fea- tures and machine learning algorithms. Recently, more and more researchers resorted to neural net- works for implicit discourse recognition (Zhang et al., 2015; Chen et al., 2016; Liu et al., 2016b; Qin et al., 2016a; Liu and Li, 2016; Braud and Denis, 2016; Wu et al., 2016). Meanwhile, to alleviate the shortage of labeled data, researcher- s explored multi-task learning with the aid of u- nannotated data for implicit discourse recognition either in traditional machine learning framework (Collobert and Weston, 2008; Lan et al., 2013) or recently in neural network framework (Wu et al., 2016; Liu et al., 2016b). In this work, we present a novel multi-task attention-based neural network to address implic- it discourse relationship representation and recog- nition. It performs two types of representation learning at the same time. An attention-based neu- ral network conducts discourse relationship repre- sentation learning through interaction between t- wo discourse arguments. Meanwhile, a multi-task learning framework leverages knowledge from auxiliary task to enhance the performance of main task. Furthermore, these two types of learning are integrated into one neural network framework and work together to maximize the overall perfor- mance. The contributions of this work are listed as fol- lows. • We propose …

Content truncated to 500 words