Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2230–2235, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition Biao Zhang1, Jinsong Su1∗, Deyi Xiong2, Yaojie Lu1, Hong Duan1 and Junfeng Yao1 Xiamen University, Xiamen, China 3610051 Soochow University, Suzhou, China 2150062 {zb, lyj}@stu.xmu.edu.cn, {jssu, hduan, yao0010}@xmu.edu.cn dyxiong@suda.edu.cn Abstract Implicit discourse relation recognition re- mains a serious challenge due to the ab- sence of discourse connectives. In this pa- per, we propose a Shallow Convolutional Neural Network (SCNN) for implicit dis- course relation recognition, which con- tains only one hidden layer but is effec- tive in relation recognition. The shallow structure alleviates the overfitting prob- lem, while the convolution and nonlinear operations help preserve the recognition and generalization ability of our model. Experiments on the benchmark data set show that our model achieves comparable and even better performance when com- paring against current state-of-the-art sys- tems. 1 Introduction As a crucial task for discourse analysis, discourse relation recognition (DRR) aims to automatically identify the internal structure and logical relation- ship of coherent text (e.g., TEMPORAL, CONTIN- GENCY, EXPANSION, etc). It provides important information to many other natural language pro- cessing systems, such as question answering (Ver- berne et al., 2007), information extraction (Cimi- ano et al., 2005), machine translation (Guzm´an et al., 2014) and so on. Despite great progress in ex- plicit DRR where the discourse connectives (e.g., “because”, “but” et al.) explicitly exist in the text (Miltsakaki et al., 2005; Pitler et al., 2008), im- plicit DRR remains a serious challenge because of the absence of discourse connectives (Prasad et al., 2008). Conventional methods for implicit DRR di- rectly rely on feature engineering, wherein re- searchers generally exploit various hand-crafted features, such as words, part-of-speech tags and ∗Corresponding author production rules (Pitler et al., 2009; Lin et al., 2009; Louis et al., 2010; Wang et al., 2012; Park and Cardie, 2012; McKeown and Biran, 2013; Lan et al., 2013; Versley, 2013; Braud and De- nis, 2014; Rutherford and Xue, 2014). Although these methods have proven successful, these man- ual features are labor-intensive and weak to cap- ture intentional, semantic and syntactic aspects that govern discourse coherence (Li et al., 2014), thus limiting the effectiveness of these methods. Recently, deep learning models have achieved remarkable results in natural language processing (Bengio et al., 2003; Bengio et al., 2006; Socher et al., 2011b; Socher et al., 2011a; Socher et al., 2013; Li et al., 2013; Kim, 2014). However, to the best of our knowledge, there is little deep learning work specifically for implicit DRR. The neglect of this important domain may be due to the follow- ing two reasons: (1) discourse relation distribution is rather unbalanced, where the generalization of deep models is relatively insufficient despite their powerful studying ability; (2) training dataset in implicit DRR is relatively small, where overfitting problems become more prominent. In this paper, we propose a Shallow Convolu- tional Neural Network (SCNN) …

Content truncated to 500 words