Intention-Based Segmentation: Human Reliability and Correlation With Linguistic Cues

INTENTION-BASED SEGMENTATION: HUMAN RELIABILITY AND CORRELATION WITH LINGUISTIC CUES Rebecca J. Passonneau Department of Computer Science Columbia University New York, NY 10027 becky@cs.columbia.edu Abstract Certain spans of utterances in a discourse, referred to here as segments, are widely assumedto form coherent units. Further, the segmental structure of discourse has been claimed to constrain and be constrained by many phenomena. However, there is weak consensus on the nature of segments and the criteria for recognizing or generating them. We present quantitative results of a two part study us- ing a corpus of spontaneous, narrative monologues. The first part evaluates the statistical reliability of human segmentation of our corpus, where speaker intention is the segmentation criterion. We then use the subjects' segmentations to evaluate the corre- lation of discourse segmentation with three linguis- tic cues (referential noun phrases, cue words, and pauses), using information retrieval metrics. INTRODUCTION A discourse consists not simply of a linear se- quence of utterances, 1 hut of meaningful relations among the utterances. As in much of the litera- ture on discourse processing, we assume that cer- tain spans of utterances, referred to here as dis- course segments, form coherent units. The seg- mental structure of discourse has been claimed to constrain and be constrained by disparate phe- nomena: cue phrases (Hirschberg and Litman, 1993; Gross and Sidner, 1986; Reichman, 1985; Co- hen, 1984); lexical cohesion (Morris and Hirst, 1991); plans and intentions (Carberry, 1990; Lit- man and Allen, 1990; Gross and Sidner, 1986); prosody (Grosz and Hirschberg, 1992; Hirschberg and Gross, 1992; Hirschberg and Pierrehumbert, 1986); reference (Webber, 1991; Gross and Sidner, 1986; Linde, 1979); and tense (Webber, 1988; Hwang and Schubert, 1992; Song and Cohen, 1991). How- ever, there is weak consensus on the nature of seg- ments and the criteria for recognizing or generat- ing them in a natural language processing system. Until recently, little empirical work has been di- rected at establishing obje'~ively verifiable segment boundaries, even though this is a precondition for 1We use the term utterance to mean a use of a sen- tence or other linguistic unit, whether in text or spoken language. Diane J. Litman AT&T Bell Laboratories 600 Mountain Avenue Murray Hill, NJ 07974 diane@research.att.com SEGMENT 1 Okay. tsk There's ~ , he looks like ay uh Chicano American, he is picking pears. A-nd u-m he's just picking them, he comes off of the ladder, a-nd he- u-h puts his pears into the basket. SEGMENT 2 U-h a number of people are going by, and one is um /you know/ I don't know, I can't remember the first ... the first person that goes by. Oh. A u-m a man with a goat comes by. It see it seems to be a busy place. You know, fairly busy, it's out in the country, maybe.in u-m u-h the valley or something. um [-~ goes up the ladder, A-nd and picks some more pears. Figure 1: Discourse Segment Structure avoiding circularity in relating segments to linguis- tic phenomena. We present the …

Content truncated to 500 words