ArchiMob - A Corpus of Spoken Swiss German

ArchiMob — A corpus of Spoken Swiss German Tanja Samardˇzi´c1, Yves Scherrer2, Elvira Glaser3 1 URPP Language and Space, University of Zurich, Freiestrasse 16, 8032 Zurich, 2 LATL-CUI, University of Geneva, Route de Drize 7, 1227 Carouge, 3 German Department, University of Zurich, Sch¨onberggasse 9, 8001 Zurich tanja.samardzic@uzh.ch, yves.scherrer@unige.ch, eglaser@ds.uzh.ch Abstract Swiss dialects of German are, unlike most dialects of well standardised languages, widely used in everyday communication. Despite this fact, automatic processing of Swiss German is still a considerable challenge due to the fact that it is mostly a spoken variety rarely recorded and that it is subject to considerable regional variation. This paper presents a freely available general-purpose corpus of spoken Swiss German suitable for linguistic research, but also for training automatic tools. The corpus is a result of a long design process, intensive manual work and specially adapted computational processing. We first describe how the documents were transcribed, segmented and aligned with the sound source, and how inconsistent transcriptions were unified through an additional normalisation layer. We then present a bootstrapping approach to automatic normalisation using different machine-translation-inspired methods. Furthermore, we evaluate the performance of part-of-speech taggers on our data and show how the same bootstrapping approach improves part-of-speech tagging by 10% over four rounds. Finally, we present the modalities of access of the corpus as well as the data format. Keywords: Swiss German, corpus, non-standard language, spoken language, normalisation, speech-to-text alignment, word level annotation 1. Introduction The term Swiss German covers a range of German varieties spoken in Switzerland on around two-thirds of its territory. Swiss German dialects are widely used in speech, while standard German is used nearly exclusively in written con- texts. This paper presents a general-purpose corpus of spo- ken Swiss German suitable for studying linguistic micro- variation and spatial diffusion with quantitative approaches, but also for developing natural language processing tools. The compilation of this corpus is set in the context of in- creasing presence of Swiss German variants in different do- mains of everyday communication. This trend results in an accumulation of language materials (TV and radio record- ing, written blogs, personal communication through vari- ous popular channels) and in an increased interest in auto- matic processing. As opposed to other, more or less digitised, sources of Swiss German data (Hotzenk¨ocherle et al., 1962-1997; Staub et al., 1881- ; Scherrer and Rambow, 2010; Kolly and Leemann, 2015), which consist of isolated word types, this corpus is intended to represent continuous speech, that is, the words as they are actually used in texts. The main difference between the corpus presented in this paper and the other two existing corpora of Swiss German, a cor- pus of SMS messages (Stark et al., 2009-2015) and a cor- pus of written texts (Hollenstein and Aepli, 2014), is the fact that this is the only corpus of transcribed spoken lan- guage. Another corpus based on transcription of spoken language is under development; it contains a smaller sam- ple of old recordings of shorter texts (more information …

Content truncated to 500 words