Representation Learning for Text-level Discourse Parsing

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 13–24, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Representation Learning for Text-level Discourse Parsing Yangfeng Ji School of Interactive Computing Georgia Institute of Technology jiyfeng@gatech.edu Jacob Eisenstein School of Interactive Computing Georgia Institute of Technology jacobe@gatech.edu Abstract Text-level discourse parsing is notoriously difficult, as distinctions between discourse relations require subtle semantic judg- ments that are not easily captured using standard features. In this paper, we present a representation learning approach, in which we transform surface features into a latent space that facilitates RST dis- course parsing. By combining the machin- ery of large-margin transition-based struc- tured prediction with representation learn- ing, our method jointly learns to parse dis- course while at the same time learning a discourse-driven projection of surface fea- tures. The resulting shift-reduce discourse parser obtains substantial improvements over the previous state-of-the-art in pre- dicting relations and nuclearity on the RST Treebank. 1 Introduction Discourse structure describes the high-level or- ganization of text or speech. It is central to a number of high-impact applications, such as text summarization (Louis et al., 2010), senti- ment analysis (Voll and Taboada, 2007; Somasun- daran et al., 2009), question answering (Ferrucci et al., 2010), and automatic evaluation of student writing (Miltsakaki and Kukich, 2004; Burstein et al., 2013). Hierarchical discourse representa- tions such as Rhetorical Structure Theory (RST) are particularly useful because of the computa- tional applicability of tree-shaped discourse struc- tures (Taboada and Mann, 2006), as shown in Fig- ure 1. Unfortunately, the performance of discourse parsing is still relatively weak: the state-of-the-art F-measure for text-level relation detection in the RST Treebank is only slightly above 55% (Joty when profit was $107.8 million on sales of $435.5 million. The projections are in the neighborhood of 50 cents a share to 75 cents, compared with a restated $1.65 a share a year earlier, CIRCUMSTANCE COMPARISON Figure 1: An example of RST discourse structure. et al., 2013). While recent work has introduced increasingly powerful features (Feng and Hirst, 2012) and inference techniques (Joty et al., 2013), discourse relations remain hard to detect, due in part to a long tail of “alternative lexicalizations” that can be used to realize each relation (Prasad et al., 2010). Surface and syntactic features are not capable of capturing what are fundamentally se- mantic distinctions, particularly in the face of rel- atively small annotated training sets. In this paper, we present a representation learn- ing approach to discourse parsing. The core idea of our work is to learn a transformation from a bag-of-words surface representation into a latent space in which discourse relations are easily iden- tifiable. The latent representation for each dis- course unit can be viewed as a discriminatively- trained vector-space representation of its meaning. Alternatively, our approach can be seen as a non- linear learning algorithm for incremental struc- ture prediction, which overcomes feature sparsity through effective parameter tying. We consider several alternative methods for transforming the original features, …

Content truncated to 500 words