Discourse Annotation in the Monroe Corpus

Discourse Annotation in the Monroe Corpus Joel Tetreault � , Mary Swift � , Preethum Prithviraj � , Myroslava Dzikovska � , James Allen � � Department of Computer Science, University of Rochester, Rochester, NY, 14620, USA tetreaul,swift,prithvir,james@cs.rochester.edu � Human Communications Research Centre, University of Edinburgh 2 Buccleuch Place, Edinburgh EH8 9LW mdzikovs@inf.ed.ac.uk Abstract We describe a method for annotating spoken dia- log corpora using both automatic and manual an- notation. Our semi-automated method for corpus development results in a corpus combining rich se- mantics, discourse information and reference anno- tation, and allows us to explore issues relating these. 1 Introduction Discourse information plays an important part in natural language systems performing tasks such as text summarization, question-answering systems and collaborative planning. But the type of dis- course information that is relevant varies widely de- pending on domain, genre, number of participants, whether it is written or spoken, etc. Therefore em- pirical analysis is necessary to determine common- alities in the variations of discourse and develop general purpose algorithms for discourse analysis. The heightened interest in human language tech- nologies in the last decade has sparked several dis- course annotation projects. Though there has been a lot of research, many of the projects focus on a few specific areas of discourse relevant to their re- spective system. For example, a text summarization system working on texts from the web would not need to know about dialogue modeling or ground- ing or prosody. In contrast, for a spoken dialogue system that collaborates with a user, such informa- tion is crucial but the organization of web pages is not. In this paper we describe our work in the Monroe Project, an effort targeting the production and use of a linguistically rich annotated corpus of a series of task-oriented spoken dialogs in an emergency res- cue domain. Our project differs from past projects involving reference annotation and discourse seg- mentation in that the semantics and discourse infor- mation is generated automatically. Most other work in this area has had minimal semantics or speech act tagging, if anything at all, which can be quite labor intensive to annotate. In addition, our domain is spoken language, which is rarely annotated for the information we are providing. We describe our research on reference resolution and discourse seg- mentation using the annotated corpus and the soft- ware tools we have developed to help us with differ- ent aspects of the annotation tasks. 2 Aims of Monroe Project 2.1 Parser Development One of the aims of the Monroe Project was to de- velop a wide coverage grammar for spoken dia- logue. Since parsing is just an initial stage of natural language understanding, the project was focused not just on obtaining syntactic trees alone (as is done in many other parsed corpora, for example, Penn TreeBank (Marcus et al., 1993) or Tiger (Brants and Plaehn, 2000)). Instead, we aimed to develop a parser and grammar for the production of syntactic parses and semantic representations useful in dis- course processing. …

Content truncated to 500 words