Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1224–1233, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention Yang Liu1,2, Sujian Li1 1 Key Laboratory of Computational Linguistics, Peking University, MOE, China 2 ILCC, School of Informatics, University of Edinburgh, United Kingdom {cs-ly, lisujian}@pku.edu.cn Abstract Recognizing implicit discourse relations is a challenging but important task in the field of Natural Language Processing. For such a complex text processing task, different from previous studies, we argue that it is necessary to repeatedly read the arguments and dynamically exploit the efficient features useful for recognizing discourse relations. To mimic the repeated reading strategy, we propose the neural networks with multi-level attention (NNMA), combining the attention mechanism and external memories to gradu- ally fix the attention on some specific words helpful to judging the discourse relations. Experiments on the PDTB dataset show that our proposed method achieves the state-of- art results. The visualization of the attention weights also illustrates the progress that our model observes the arguments on each level and progressively locates the important words. 1 Introduction Discourse relations (e.g., contrast and causality) support a set of sentences to form a coherent text. Automatically recognizing discourse relations can help many downstream tasks such as question answering and automatic summarization. Despite great progress in classifying explicit discourse relations where the discourse connectives (e.g., “because”, “but”) explicitly exist in the text, implicit discourse relation recognition remains a challenge due to the absence of discourse connec- tives. Previous research mainly focus on exploring various kinds of efficient features and machine learning models to classify the implicit discourse relations (Soricut and Marcu, 2003; Baldridge and Lascarides, 2005; Subba and Di Eugenio, 2009; Hernault et al., 2010; Pitler et al., 2009; Joty et al., 2012). To some extent, these methods simulate the single-pass reading process that a person quickly skim the text through one-pass reading and directly collect important clues for understanding the text. Although single-pass reading plays a crucial role when we just want the general meaning and do not necessarily need to understand every single point of the text, it is not enough for tackling tasks that need a deep analysis of the text. In contrast with single-pass reading, repeated reading involves the process where learners repeatedly read the text in detail with specific learning aims, and has the potential to improve readers’ reading fluency and comprehension of the text (National Institute of Child Health and Human Development, 2000; LaBerge and Samuels, 1974). Therefore, for the task of discourse parsing, repeated reading is necessary, as it is difficult to generalize which words are really useful on the first try and efficient features should be dynamically exploited through several passes of reading . Now, let us check one real example to elaborate the necessity of using repeated reading in discourse parsing. Arg-1 : the use of 900 toll numbers has been expanding rapidly in recent years Arg-2 : …

Content truncated to 500 words