Deep Enhanced Representation for Implicit Discourse Relation Recognition

Proceedings of the 27th International Conference on Computational Linguistics, pages 571–583 Santa Fe, New Mexico, USA, August 20-26, 2018. 571 Deep Enhanced Representation for Implicit Discourse Relation Recognition Hongxiao Bai1,2, Hai Zhao1,2,∗ 1Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China 2Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China baippa@sjtu.edu.cn, zhaohai@cs.sjtu.edu.cn Abstract Implicit discourse relation recognition is a challenging task as the relation prediction without explicit connectives in discourse parsing needs understanding of text spans and cannot be easily derived from surface features from the input sentence pairs. Thus, properly representing the text is very crucial to this task. In this paper, we propose a model augmented with different grained text representations, including character, subword, word, sentence, and sentence pair levels. The proposed deeper model is evaluated on the benchmark treebank and achieves state- of-the-art accuracy with greater than 48% in 11-way and F1 score greater than 50% in 4-way classifications for the first time according to our best knowledge. 1 Introduction Discourse parsing is a fundamental task in natural language processing (NLP) which determines the structure of the whole discourse and identifies the relations between discourse spans such as clauses and sentences. Improving this task can be helpful to many downstream tasks such as machine trans- lation (Li et al., 2014), question answering (Jansen et al., 2014), and so on. As one of the important parts of discourse parsing, implicit discourse relation recognition task is to find the relation between two spans without explicit connectives (e.g., but, so, etc.), and needs recovering the relation from semantic understanding of texts. The Penn Discourse Treebank 2.0 (PDTB 2.0) (Prasad et al., 2008) is a benchmark corpus for discourse relations. In PDTB style, the connectives can be explicit or implicit, and one entry of the data is separated into Arg1 and Arg2, accompanied with a relation sense. Since the release of PDTB 2.0 dataset, many methods have been proposed, ranging from traditional feature-based methods (Lin et al., 2009; Pitler et al., 2009) to latest neural-based methods (Qin et al., 2017; Lan et al., 2017). Especially through many neural network methods used for this task such as convolutional neural network (CNN) (Qin et al., 2016b), recursive neural network (Ji and Eisenstein, 2015), embedding improvement (Wu et al., 2017), attention mechanism (Liu and Li, 2016), gate mechanism (Chen et al., 2016), multi-task method (Lan et al., 2017), the performance of this task has improved a lot since it was first introduced. However, this task is still very challenging with the highest reported accuracy still lower than 50% due to the hardness for the machines to understand the text meaning and the relatively small task corpus. In this work, we focus on improving the learned representations of sentence pairs to address the im- plicit discourse relation recognition. It is well known that text representation is the core part of state-of- the-art deep learning methods for NLP tasks, and improving the representation from …

Content truncated to 500 words