Distilling dialogues - A method using natural dialogue corpora for dialogue systems development

Distilling dialogues - A method using natural dialogue dialogue systems development Arne JSnsson and Nils Dahlb~ick Department of Computer and Information Science LinkSping University S-581 83, LINKOPING SWEDEN nilda@ida.liu.se, arnjo@ida.liu.se corpora for Abstract We report on a method for utilising corpora col- lected in natural settings. It is based on distilling (re-writing) natural dialogues to elicit the type of dialogue that would occur if one the dialogue par- ticipants was a computer instead of a human. The method is a complement to other means such as Wiz- ard of Oz-studies and un-distilled natural dialogues. We present the distilling method and guidelines for distillation. We also illustrate how the method af- fects a corpus of dialogues and discuss the pros and cons of three approaches in different phases of dia- logue systems development. 1 Introduction It has been known for quite some time now, that the language used when interacting with a comput- er is different from the one used in dialogues between people, (c.f. JSnsson and Dahlb~ick (1988)). Given that we know that the language will be different, but not how it will be different, we need to base our development of natural language dialogue sys- tems on a relevant set of dialogue corpora. It is our belief that we need to clarify a number of different issues regarding the collection and use of corpora in the development of speech-only and multimodal dia- logue systems. Exchanging experiences and develop- ing guidelines in this area are as important as, and in some sense a necessary pre-requisite to, the develop- ment of computational models of speech, language, and dialogue/discourse. It is interesting to note the difference in the state of art in the field of natu- ral language dialogue systems with that of corpus linguistics, where issues of the usefulness of different samples, the necessary sampling size, representative- ness in corpus design and other have been discussed for quite some time (e.g. (Garside et al., 1997; Atkins et al., 1992; Crowdy, 1993; Biber, 1993)). Also the neighboring area of evaluation of NLP systems (for an overview, see Sparck Jones and Galliers (1996)) seems to have advanced further. Some work have been done in the area of natu- ral language dialogue systems, e.g. on the design of Wizard of Oz-studies (Dahlb~ck et al., 1998), on measures for inter-rater reliability (Carletta, 1996), on frameworks for evaluating spoken dialogue agents (Walker et al., 1998) and on the use of differ- ent corpora in the development of a particular sys- tem (The Carnegie-Mellon Communicator, Eskenazi et al. (1999)). The question we are addressing in this paper is how to collect and analyse relevant corpora. We be- gin by describing what we consider to be the main advantages and disadvantages of the two currently used methods; studies of human dialogues and Wiz- ard of Oz-dialogues, especially focusing on the eco- logical validity of the methods. We then describe a method called 'distilling dialogues', which can serve as a supplement to the other two. 2 Natural and Wizard …

Content truncated to 500 words