Combining Multiple Knowledge Sources for Discourse Segmentation

Combining Multiple Knowledge Sources for Discourse Segmentation Diane J. Litman AT&T Bell Laboratories 600 Mountain Avenue Murray Hill, NJ 07974 diane@research.att.com Rebecca J. Passonneau* Bellcore 445 South Street Morristown, NJ 07960 beck~bellcore.com Abstract We predict discourse segment boundaries from linguistic features of utterances, using a corpus of spoken narratives as data. We present two methods for developing seg- mentation algorithms from training data: hand tuning and machine learning. When multiple types of features are used, results approach human performance on an inde- pendent test set (both methods), and using cross-validation (machine learning). 1 Introduction Many have argued that discourse has a global struc- ture above the level of individual utterances, and that linguistic phenomena like prosody, cue phra- ses, and nominal reference are partly conditioned by and reflect this structure (cf. (Grosz and Hirschberg, 1992; Grosz and Sidner, 1986; Hirschberg and Grosz, 1992; Hirschberg and Litman, 1993; Hirschberg and Pierrehumbert, 1986; Hobbs, 1979; Lascarides and Oberlander, 1992; Linde, 1979; Mann and Thomp- son, 1988; Polanyi, 1988; Reichman, 1985; Webber, 1991)). However, an obstacle to exploiting the rela- tion between global structure and linguistic devices in natural language systems is that there is too little data about how they constrain one another. We have been engaged in a study addressing this gap. In previous work (Passonneau and Litman, 1993), we reported on a method for empirically validating global discourse units, and on our evaluation of algo- rithms to identify these units. We found significant agreement among naive subjects on a discourse seg- mentation task, which suggests that global discourse units have some objective reality. However, we also found poor correlation of three untuned algorithms (based on features of referential noun phrases, cue words, and pauses, respectively) with the subjects' segmentations. In this paper, we discuss two methods for develo- ping segmentation algorithms using multiple know- *Bellcore did not support the second author's work. ledge sources. In section 2, we give a brief overview of related work and summarize our previous results. In section 3, we discuss how linguistic features are coded and describe our evaluation. In section 4, we present our analysis of the errors made by the best performing untuned algorithm, and a new algorithm that relies on enriched input features and multiple knowledge sources. In section 5, we discuss our use of machine learning tools to automatically construct decision trees for segmentation from a large set of input features. Both the hand tuned and automa- tically derived algorithms improve over our previ- ous algorithms. The primary benefit of the hand tuning is to identify new input features for impro- ving performance. Machine learning tools make it convenient to perform numerous experiments, to use large feature sets, and to evaluate results using cross- validation. We discuss the significance of our results and briefly compare the two methods in section 6. 2 Discourse Segmentation 2.1 Related Work Segmentation has played a significant role in much work on discourse. The linguistic structure of Grosz and Sidner's (1986) tri-partite discourse model con- sists …

Content truncated to 500 words