Neural Generative Rhetorical Structure Parsing

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2284–2295, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2284 Neural Generative Rhetorical Structure Parsing Amandla Mabona Dept. of Computer Science and Technology University of Cambridge amandla.mabona@cl.cam.ac.uk Laura Rimell DeepMind London, UK laurarimell@google.com Stephen Clark DeepMind London, UK clarkstephen@google.com Andreas Vlachos Dept. of Computer Science and Technology University of Cambridge andreas.vlachos@cst.cam.ac.uk Abstract Rhetorical structure trees have been shown to be useful for several document-level tasks in- cluding summarization and document classi- fication. Previous approaches to RST pars- ing have used discriminative models; however, these are less sample efficient than generative models, and RST parsing datasets are typically small. In this paper, we present the first gen- erative model for RST parsing. Our model is a document-level RNN grammar (RNNG) with a bottom-up traversal order. We show that, for our parser’s traversal order, previous beam search algorithms for RNNGs have a left-branching bias which is ill-suited for RST parsing. We develop a novel beam search algorithm that keeps track of both structure- and word-generating actions without exhibit- ing this branching bias and results in absolute improvements of 6.8 and 2.9 on unlabelled and labelled F1 over previous algorithms. Overall, our generative model outperforms a discrimi- native model with the same features by 2.6 F1 points and achieves performance comparable to the state-of-the-art, outperforming all pub- lished parsers from a recent replication study that do not use additional training data. 1 Introduction Understanding a document’s discourse-level or- ganization is important for correctly interpreting it, and discourse analyses have been shown to be helpful for several NLP tasks (Bhatia et al., 2015; Ji and Smith, 2017; Feng and Hirst, 2014b; Fer- racane et al., 2017). A popular formalism for discourse analysis is Rhetorical Structure The- ory (RST) (Mann and Thompson, 1988) (Fig. 1) which represents a document as a tree of discourse units recursively built by connecting smaller units through rhetorical relations. Learning to predict RST trees is difficult because it depends on prag- matics as well as literal meaning, and the En- glish RST Discourse Treebank (RST-DT) (Carlson et al., 2003) is small by the standards of modern parsing datasets, with 347 training documents. Previous approaches to RST parsing (Ji and Eisenstein, 2014; Feng and Hirst, 2014a; Joty et al., 2015; Braud et al., 2017) have used lo- cally normalized discriminative models. However, these are known to have worse performance than generative models when there is little training data (Ng and Jordan, 2002; Yogatama et al., 2017). Unlike locally normalised discriminative mod- els, generative models are not susceptible to label bias (Lafferty et al., 2001). The success of gener- ative (Dyer et al., 2016; Charniak et al., 2016) and globally normalised (Andor et al., 2016) syntactic parsers suggests that reducing label bias leads to better performance. We hypothesize that using a generative parser would also lead to improved per- formance on RST parsing. However, while they …

Content truncated to 500 words