Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale Pretraining

Proceedings of the 28th International Conference on Computational Linguistics, pages 3794–3805 Barcelona, Spain (Online), December 8-13, 2020 3794 Unleashing the Power of Neural Discourse Parsers – A Context and Structure Aware Approach Using Large Scale Pretraining Grigorii Guz, Patrick Huber and Giuseppe Carenini Department of Computer Science University of British Columbia Vancouver, BC, Canada, V6T 1Z4 {gguz, huberpat, carenini}@cs.ubc.ca Abstract RST-based discourse parsing is an important NLP task with numerous downstream applications, such as summarization, machine translation and opinion mining. In this paper, we demonstrate a simple, yet highly accurate discourse parser, incorporating recent contextual language mod- els. Our parser establishes the new state-of-the-art (SOTA) performance for predicting structure and nuclearity on two key RST datasets, RST-DT and Instr-DT. We further demonstrate that pretraining our parser on the recently available large-scale “silver-standard” discourse treebank MEGA-DT provides even larger performance benefits, suggesting a novel and promising re- search direction in the field of discourse analysis. 1 Introduction Discourse parsing is an important upstream task within the area of Natural Language Processing (NLP) which has been an active field of research over the last decades. In this work, we focus on discourse representations for the English language, where most research has been surrounding one of the two main theories behind discourse, the Rhetorical Structure Theory (RST) proposed by Mann and Thompson (1988) or interpreting discourse according to PDTB (Prasad et al., 2008). While both theories have their strengths, the application of the RST theory, encoding documents into complete constituency discourse trees (Morey et al., 2018), has been shown to have many crucial implications on real world problems. A tree is defined on a set of EDUs (Elementary Discourse Units), approximately aligning with clause-like sentence fragments, acting as the leaves of the tree. Adjacent EDUs or sub-trees are hierarchically aggre- gated to form larger (possibly non-binary) constituents, with internal nodes containing (1) a nuclearity label, defining the importance of the subtree (rooted at the internal node) in the local context and (2) a relation label, defining the type of semantic connection between the two subtrees (e.g., Elaboration, Background). In this work, we focus on structure and nuclearity prediction, not taking relations into account. Previous research has shown that the use of RST-style discourse parsing as a system component can enhance important tasks, such as sentiment analysis, summarization and text categorization (Bhatia et al., 2015; Nejat et al., 2017; Hogenboom et al., 2015; Gerani et al., 2014; Ji and Smith, 2017). More recently, it has also been suggested that discourse structures obtained in an RST-style manner can further be complementary to learned contextual embeddings, like the popular BERT approach (Devlin et al., 2018). Combining both approaches has shown to support tasks where linguistic information on complete documents is critical, such as argumentation analysis (Chakrabarty et al., 2019). Even though discourse parsers appear to enhance the performance on a variety of tasks, the full potential of using more lin- guistically inspired approaches for downstream applications has not been unleashed yet. The main open challenges of …

Content truncated to 500 words