Representing discourse coherence: A corpus-based analysis

Representing discourse coherence: A corpus-based analysis Florian WOLF MIT NE20-448 Cambridge, MA 02139, USA fwolf@mit.edu Edward GIBSON MIT NE20-459 Cambridge, MA, 02139, USA egibson@mit.edu Abstract We present a set of discourse structure relations that are easy to code, and develop criteria for an appropriate data structure for representing these relations. Discourse structure here refers to informational relations that hold between sentences in a discourse (cf. Hobbs, 1985). We evaluated whether trees are a descriptively adequate data structure for representing coherence. Trees are widely assumed as a data structure for representing coherence but we found that more powerful data structures are needed: In coherence structures of naturally occurring texts, we found many different kinds of crossed dependencies, as well as many nodes with multiple parents. The claims are supported by statistical results from a database of 135 texts from the Wall Street Journal and the AP Newswire that were hand-annotated with coherence relations, based on the annotation schema presented in this paper. 1 Introduction An important component of natural language discourse understanding and production is having a representation of discourse structure. A coherently structured discourse here is assumed to be a collection of sentences that are in some relation to each other. This paper aims to present a set of discourse structure relations that are easy to code, and to develop criteria for an appropriate data structure for representing these relations. Discourse structure relations here refer to informational relations that hold between sentences or other non-overlapping segments in a discourse monologue. That is, discourse structure relations reflect how the meaning conveyed by one discourse segment relates to the meaning conveyed by another discourse segment (cf. Hobbs, 1985; Marcu, 2000; Webber et al., 1999). Accounts of discourse structure vary greatly with respect to how many discourse relations they assume, ranging from two (Grosz & Sidner, 1986) to over 400 different coherence relations, reported in Hovy and Maier (1995). However, Hovy and Maier (1995) argue that taxonomies with more relations represent subtypes of taxonomies with fewer relations. This means that different taxonomies can be compatible with each other. We describe an account with a small number of relations in order to achieve more generalizable representations of discourse structures; however, the number is not so small that informational structures that we are interested in are obscured. The next section will describe in detail the set of coherence relations we use, which are mostly based on Hobbs (1985). Additionally, we try to make as few a priori theoretical assumptions about representational data structures as possible. These assumptions will be outlined in the next section. Importantly, however, we do not assume a tree data structure to represent discourse coherence structures. In fact, a major goal of this paper is to show that trees do not seem adequate to represent discourse structures. 2 Collecting a database of texts annotated with coherence relations This section describes (1) how we define discourse segments, (2) which coherence relations we used to connect the discourse segments, and (3) how the annotation …

Content truncated to 500 words