DUEL: A Multi-lingual Multimodal Dialogue Corpus for Disfluency, Exclamations and Laughter

DUEL: A Multi-lingual Multimodal Dialogue Corpus for Disfluency, Exclamations and Laughter Julian Hough1, Ye Tian2, Laura de Ruiter3, Simon Betz1, Spyros Kousidis4, David Schlangen1, Jonathan Ginzburg2 1Dialogue Systems Group, Bielefeld University, 2Laboratoire de linguistique formelle, Universit´e Paris-Diderot, 3School of Psychological Sciences, University of Manchester, 4Carmeq GmbH julian.hough@uni-bielefeld.de Abstract We present the DUEL corpus, consisting of 24 hours of natural, face-to-face, loosely task-directed dialogue in German, French and Mandarin Chinese. The corpus is uniquely positioned as a cross-linguistic, multimodal dialogue resource controlled for domain. DUEL includes audio, video and body tracking data and is transcribed and annotated for disfluency, laughter and exclamations. Keywords: disfluency, laughter, exclamations, multi-lingual, multimodal, spontaneous 1. Introduction Natural, spontaneous dialogue corpora are rich resources for a variety of linguistic research. In this paper, we present the DUEL (‘Disfluency, exclamations and laughter in dialogue’ (Ginzburg et al., 2014b)) corpus,1 consisting of 24 hours of natural, face-to-face, loosely task-directed dialogue in German, French and Mandarin Chinese. The corpus is uniquely positioned as a cross-linguistic, mul- timodal dialogue resource controlled for domain, including audio, video and body tracking data and is transcribed and annotated for disfluency, laughter and exclamations. To ensure cross-linguistic comparability, the experimental tasks were designed to be culture-neutral, the data in three languages were recorded using near-identical technical se- tups, and our transcription and annotation protocol is de- signed to be language-general. In this paper, we give a summary of the tasks, the recording procedure and the transcription and annotation protocol. Then we discuss briefly the characteristics of our corpus, possible use cases and implications for natural dialogue research. 2. Existing Spontaneous Speech Corpora in Target Languages Previous corpus work on spontaneous speech in German has focused on small domains and/or on speech data that does not generalize well to natural face-to-face dialogue. Kohler (1996) elicited dialogues using an appointment making sce- nario, but had speakers press a button to speak, eliminating any turn-overlaps (and potential disfluencies resulting from these). This is similar to (Burger et al., 2000), who used a similar scenario and instructed speakers not to interrupt each other. Schiel et al. (2012)’s non-intoxicated sponta- neous control data was obtained by having participants talk to the experimenter in a car. Schmidt et al. (2010) and the 1The DUEL project website is at: http://www.dsg-bielefeld.de/DUEL Berlin Map Task Corpus (BeMaTaC)2 both used map tasks, with the latter recording only non-native speakers. Peters (2005) collected a corpus of spontaneous speech by having two friends talk about video sequences via headset without eye-contact. For French, there are several corpora for spontaneous speech. Several projects collected spoken French for study- ing prosody, for example, PFC (Durand et al., 2009), C- PROM (Avanzi et al., 2010) and Rhapsodie (Lacheret et al., 2014). Because of their research interests, these corpora cover a variety of discourse genres and do not focus on face- to-face dialogues. Bonneau-Maynard et al. (2005) collected the MEDIA corpus, containing roughly 70 hours of French dialogues on the topics of tourist information inquiry and hotel booking. It was recorded …

Content truncated to 500 words