A Knowledge-Augmented Neural Network Model for Implicit Discourse Relation Classification

Proceedings of the 27th International Conference on Computational Linguistics, pages 584–595 Santa Fe, New Mexico, USA, August 20-26, 2018. 584 A Knowledge-Augmented Neural Network Model for Implicit Discourse Relation Classification Yudai Kishimoto Yugo Murawaki Sadao Kurohashi Graduate School of Informatics, Kyoto University Yoshida-honmachi, Sakyo-ku, Kyoto, 606-8501, Japan {kishimoto, murawaki, kuro}@nlp.ist.i.kyoto-u.ac.jp Abstract Identifying discourse relations that are not overtly marked with discourse connectives remains a challenging problem. The absence of explicit clues indicates a need for the combination of world knowledge and weak contextual clues, which can hardly be learned from a small amount of manually annotated data. In this paper, we address this problem by augmenting the input text with external knowledge and context and by adopting a neural network model that can effec- tively handle the augmented text. Experiments show that external knowledge did improve the classification accuracy. On the other hand, contextual information provided no significant gain for implicit discourse relations while it worked for explicit ones. 1 Introduction Discourse relation recognition has a wide variety of potential applications including summariza- tion (Louis et al., 2010), sentiment analysis (Somasundaran et al., 2009) and machine translation (Meyer et al., 2015). In one of the two most prevalent discourse treebanks, the Penn Discourse TreeBank (PDTB) (Prasad et al., 2008), discourse relations are conventionally divided into two types: explicit and implicit. Explicit relations are overtly marked with discourse connectives such as “because” and “however.” Because of these strong cues, explicit relations are relatively easy to classify (Pitler et al., 2008; Xue et al., 2016). By contrast, implicit relations lack discourse connectives and classifying such relations remains a challenging problem. Recent studies on implicit discourse relation classification have shown success in applying various neural network models including feedforward networks (Zhang et al., 2015; Schenk et al., 2016), convo- lutional neural networks (Mihaylov and Frank, 2016; Wang and Lan, 2016) and bidirectional LSTM (bi- LSTM) (Chen et al., 2016; Liu and Li, 2016; Dai and Huang, 2018). Although these studies on network engineering report performance improvement, Rutherford et al. (2017) demonstrated that a simple feed- forward neural network was astonishingly competitive, outperforming LSTM- and Tree LSTM-based models. He claimed that training data for implicit discourse relation classification were too small to train powerful neural networks like LSTM. This motivates us to view this task from a different perspective. We argue that the neural network models need to be provided with world knowledge, which can hardly be learned from a small amount of manually annotated data. To see this, suppose that we want to classify the discourse relation of the following pair of text spans (referred to as Arg1 (in italic) and Arg2 (in bold) throughout this paper):   Arg1: Not counting the extraordinary charge it would have had a net loss of $3.1 million, or seven cents a share Arg2: A year earlier, it had profit of $7.5 million, or 18 cents a share (Comparison.Contrast)   We can easily recognize that the discourse relation between this pair is Comparison.Contrast partly be- …

Content truncated to 500 words