A Novel Discourse Parser Based on Support Vector Machine Classification

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 665–673, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP A Novel Discourse Parser Based on Support Vector Machine Classification David A. duVerle National Institute of Informatics Tokyo, Japan Pierre & Marie Curie University Paris, France dave@nii.ac.jp Helmut Prendinger National Institute of Informatics Tokyo, Japan helmut@nii.ac.jp Abstract This paper introduces a new algorithm to parse discourse within the framework of Rhetorical Structure Theory (RST). Our method is based on recent advances in the field of statistical machine learning (mul- tivariate capabilities of Support Vector Machines) and a rich feature space. RST offers a formal framework for hierarchical text organization with strong applications in discourse analysis and text generation. We demonstrate automated annotation of a text with RST hierarchically organised relations, with results comparable to those achieved by specially trained human anno- tators. Using a rich set of shallow lexical, syntactic and structural features from the input text, our parser achieves, in linear time, 73.9% of professional annotators’ human agreement F-score. The parser is 5% to 12% more accurate than current state-of-the-art parsers. 1 Introduction According to Mann and Thompson (1988), all well-written text is supported by a hierarchically structured set of coherence relations which reflect the authors intent. The goal of discourse parsing is to extract this high-level, rhetorical structure. Dependency parsing and other forms of syn- tactic analysis provide information on the gram- matical structure of text at the sentential level. Discourse parsing, on the other hand, focuses on a higher-level view of text, allowing some flexibility in the choice of formal representation while providing a wide range of applications in both analytical and computational linguistics. Rhetorical Structure Theory (Mann and Thomp- son, 1988) provides a framework to analyze and study text coherence by defining and applying a set of structural relations to composing units (‘spans’) of text. Annotation of a text within the RST formalism will produce a tree-like structure that not only reflects text-coherence but also provides input for powerful algorithmic tools for tasks such as text regeneration (Piwek et al., 2007). RST parsing can be seen as a two-step process: 1. Segmentation of the input text into elemen- tary discourse units (‘edus’). 2. Generation of the rhetorical structure tree based on ‘rhetorical relations’ (or ‘coherence relations’) as labels of the tree, with the edus constituting its terminal nodes. Mann and Thompson (1988) empirically estab- lished 110 distinct rhetorical relations, but pointed out that this set was flexible and open-ended. In addition to rhetorical relations, RST defines the notion of ‘nucleus’, the relatively more important part of the text, and ‘satellite’, which is subordinate to the nucleus. In Fig. 1, the left- most edu constitutes the satellite (indicated by out-going arrow), and the right-hand statement constitutes the nucleus. Observe that the nucleus itself is a compound of nucleus and satellite. Several attempts to automate discourse parsing have been made. Marcu and Soricut focussed on sentence-level parsing and developed two probabilistic …

Content truncated to 500 words