Creating Annotated Dialogue Resources: Cross-domain Dialogue Act Classification

Creating Annotated Dialogue Resources: Cross-Domain Dialogue Act Classification Dilafruz Amanova1, Volha Petukhova2, Dietrich Klakow2 1Max Planck Institute for Informatics, Saarbr¨ucken, Germany 2Saarland University, Spoken Language Systems, Saarbr¨ucken, Germany 1dilafruz@mpi-inf.mpg.de; 2{v.petukhova,dietrich.klakow}@lsv.uni-saarland.de Abstract This paper describes a method to automatically create dialogue resources annotated with dialogue act information by reusing existing dialogue corpora. Numerous dialogue corpora are available for research purposes and many of them are annotated with dialogue act information that captures the intentions encoded in user utterances. Annotated dialogue resources, however, differ in various respects: data collection settings and modalities used, dialogue task domains and scenarios (if any) underlying the collection, number and roles of dialogue participants involved and dialogue act annotation schemes applied. The presented study encompasses three phases of data-driven investigation. We, first, assess the importance of various types of features and their combinations for effective cross-domain dialogue act classification. Second, we establish the best predictive model comparing various cross-corpora training settings. Finally, we specify models adaptation procedures and explore late fusion approaches to optimize the overall classification decision taking process. The proposed methodology accounts for empirically motivated and technically sound classification procedures that may reduce annotation and training costs significantly. Keywords: cross-domain dialogue act classification, ISO 24617-2 annotated resources, dialogue act model adaptation, meta- classification 1. Introduction Corpora annotated with semantic information gained lots of researchers’ and practitioners’ interest and are acknowl- edged to be important for a wide range of linguistic appli- cations. In the dialogue research area, it became common to annotate dialogue corpus data with dialogue act infor- mation. Dialogue corpus annotations may serve various purposes. Annotated data is used for a systematic analy- sis of a variety of dialogue phenomena, such as turn-taking, feedback, and recurring structural patterns. Corpus data annotated with dialogue act information are also used to train machine learning algorithms for the automatic recog- nition and prediction of dialogue acts as a part of a human- machine dialogue system. The dialogue research community still does not have large amounts of annotated dialogue data at its disposal as com- pared to other linguistic communities. Although a lot of work has been done in this direction, e.g. the HCRC MapTask corpus (Carletta et al., 1996), the AMI1 and ICSI-MRDA (Dhillon et al., 2004) meeting corpora, the Switchboard-DAMSL (Jurafsky, 1997) and Coconut (Di Eugenio et al., 1998) corpora, just to name few. The avail- able resources however are only partly compatible with each other. Annotations are based on schemes developed for analysis and modelling corpus- and domain-specific di- alogue behaviour, and annotation formats vary a lot ranging from plain text to XML based representations, e.g. SGML and NXT. Annotations differ with respect to the range of phenomena they cover, their granularity level of defined concepts, segmentation principles and their theory depen- dencies. Thus, such corpora are not easy to re-use for purposes and apply to domains other than they were origi- nally developed for. For cross-domain analysis it is impor- 1http://groups.inf.ed.ac.uk/ami/corpus/ tant to have interoperable annotated resources. One way to achieve interoperability is …

Content truncated to 500 words