The Rhetorical Parsing of Unrestricted Natural Language Texts

The Rhetorical Parsing of Natural Language Texts Daniel Marcu Department of Computer Science University of Toronto Toronto, Ontario Canada M5S 3G4 marcu~cs, toronto, edu Abstract We derive the rhetorical structures of texts by means of two new, surface-form-based algorithms: one that identifies discourse usages of cue phrases and breaks sen- tences into clauses, and one that produces valid rhetorical structure trees for unre- stricted natural language texts. The algo- rithms use information that was derived from a corpus analysis of cue phrases. 1 Introduction Researchers of natural language have repeatedly ac- knowledged that texts are not just a sequence of words nor even a sequence of clauses and sentences. However, despite the impressive number of discourse-related theo- ries that have been proposed so far, there have emerged no algorithms capable of deriving the discourse struc- ture of an unrestricted text. On one hand, efforts such as those described by Asher (1993), Lascarides, Asher, and Oberlander (1992), Kamp and Reyle (1993), Grover et al. (1994), and Pr0st, Scha, and van den Berg (1994) take the position that discourse structures can be built only in conjunction with fully specified clause and sen- tence structures. And Hobbs's theory (1990) assumes that sophisticated knowledge bases and inference mech- anisms are needed for determining the relations between discourse units. Despite the formal elegance of these approaches, they are very domain dependent and, there- fore, unable to handle more than a few restricted exam- pies. On the other hand, although the theories described by Grosz and Sidner (1986), Polanyi (1988), and Mann and Thompson (1988) are successfully applied manually, they ,are too informal to support an automatic approach to discourse analysis. In contrast with this previous work, the rhetorical parser that we present builds discourse trees for unre- stricted texts. We first discuss the key concepts on which our approach relies (section 2) and the corpus analysis (section 3) that provides the empirical data for our rhetor- ical parsing algorithm. We discuss then an algorithm that recognizes discourse usages of cue phrases and that de- termines clause boundaries within sentences. Lastly, we present the rhetorical parser and an example of its opera- tion (section 4). 2 Foundation The mathematical foundations of the rhetorical parsing algorithm rely on a first-order formalization of valid text structures (Marcu, 1997). The assumptions of the for- malization are the following. 1. The elementary units of complex text structures are non-overlapping spans of text. 2. Rhetorical, coherence, and cohesive relations hold between textual units of various sizes. 3. Rela- tions can be partitioned into two classes: paratactic and hypotactic. Paratactic relations are those that hold be- tween spans of equal importance. Hypotactic relations are those that hold between a span that is essential for the writer's purpose, i.e., a nucleus, and a span that increases the understanding of the nucleus but is not essential for the writer's purpose, i.e., a satellite. 4. The abstract structure of most texts is a binary, tree-like structure. 5. If a relation holds between two textual spans …

Content truncated to 500 words