Implicit Discourse Relation Recognition using Neural Tensor Network with Interactive Attention and Sparse Learning

Proceedings of the 27th International Conference on Computational Linguistics, pages 547–558 Santa Fe, New Mexico, USA, August 20-26, 2018. 547 Implicit Discourse Relation Recognition using Neural Tensor Network with Interactive Attention and Sparse Learning Fengyu Guo1,2,∗, Ruifang He1,2,∗,†, Di Jin1,2, Jianwu Dang1,2,3, Longbiao Wang1,2, Xiangang Li4 1School of Computer Science and Technology, Tianjin University, Tianjin, China. 2Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin, China. 3Japan Advanced Institute of Science and Technology, Ishikawa, Japan. 4AI Labs, Didi Chuxing, Beijing, China. {fengyuguo,rfhe,jindi,longbiao wang}@tju.edu.cn jdang@jaist.ac.jp, lixiangang@didichuxing.com Abstract Implicit discourse relation recognition aims to understand and annotate the latent relations be- tween two discourse arguments, such as temporal, comparison, etc. Most previous methods encode two discourse arguments separately, the ones considering pair specific clues ignore the bidirectional interactions between two arguments and the sparsity of pair patterns. In this paper, we propose a novel Neural Tensor Network framework with Interactive Attention and Sparse Learning (TIASL) for implicit discourse relation recognition. (1) We mine the most correlated word pairs from two discourse arguments to model pair specific clues, and integrate them as in- teractive attention into argument representations produced by the bidirectional long short-term memory network. Meanwhile, (2) the neural tensor network with sparse constraint is proposed to explore the deeper and the more important pair patterns so as to fully recognize discourse rela- tions. The experimental results on PDTB show that our proposed TIASL framework is effective. 1 Introduction Discourse relation describes how two adjacent text units (e.g. clauses, sentences, and larger sentence groups), called arguments, named Arg1 and Arg2, are connected semantically, such as temporally, causally, etc. Yet implicit discourse relation recognition without explicit connectives (Pitler et al., 2008), which needs to infer the relation from specific context, is still a challenging problem. It can be used in text summarization(Gerani et al., 2014), conversation system (Higashinaka et al., 2014) and so on. Previous researches mainly include (1) traditional feature-based models and (2) neural network based models. Most feature-based models adopt various linguistic features (such as polarity, word pairs, and position information, etc.) and design complicated rules to recognize implicit discourse relations (Pitler et al., 2009; Zhou et al., 2010; Braud and Denis, 2015). They can not fully use the local and the global context, and the human cost is huge. Neural network based models get the better argument representations and more precisely capture discourse relations (Braud and Denis, 2015; Zhang et al., 2015; Liu et al., 2016). However, they encode two discourse arguments separately, and ignore pair specific clues. The further researches adopt the different hybrid neural models (Chen et al., 2016; Lei et al., 2017) and attention mechanism (Cai and Zhao, 2017) to mine the semantic interactions of argument pairs. Yet, they ignore the bidirectional interactions between two arguments during the representation stage since there is asymmetry from the perspective of human-like reading strategy. And the sparsity of word pair patterns indicating discourse relation is neither considered. Therefore, a novel neural Tensor network model with Interactive Attention and Sparse Learning …

Content truncated to 500 words