A Swiss German Dictionary: Variation in Speech and Writing

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2720–2725 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2720 A Swiss German Dictionary: Variation in Speech and Writing Larissa Schmidt∗, Lucy Linder†, Sandra Djambazovska††, Alexandros Lazaridis††, Tanja Samardˇzi´c∗, Claudiu Musat†† ∗ University of Zurich, URPP Language and Space † University of Fribourg †† Data, Analytics and AI (DNA) - Swisscom AG ∗ {larissa.schmidt, tanja.samardzic}@uzh.ch, † lucy.linder@hefr.ch, †† sandra.djambazovska@gmail.com, {alexandros.lazaridis, claudiu.musat}@swisscom.com Abstract We introduce a dictionary containing forms of common words in various Swiss German dialects normalized into High German. As Swiss German is, for now, a predominantly spoken language, there is a significant variation in the written forms, even between speakers of the same dialect. To alleviate the uncertainty associated with this diversity, we complement the pairs of Swiss German - High German words with the Swiss German phonetic transcriptions (SAMPA). This dictionary becomes thus the first resource to combine large-scale spontaneous translation with phonetic transcriptions. Moreover, we control for the regional distribution and insure the equal representation of the major Swiss dialects. The coupling of the phonetic and written Swiss German forms is powerful. We show that they are sufficient to train a Transformer-based phoneme to grapheme model that generates credible novel Swiss German writings. In addition, we show that the inverse mapping - from graphemes to phonemes - can be modeled with a transformer trained with the novel dictionary. This generation of pronunciations for previously unknown words is key in training extensible automated speech recognition (ASR) systems, which are key beneficiaries of this dictionary. Keywords: NLP, Language Modeling, G2P, Machine Translation, Swiss German, Speech, Non-standard 1. Introduction Swiss German refers to any of the German varieties that are spoken in about two thirds of Switzerland (Samardˇzi´c et al., 2016). Besides at least one of those dialectal varieties, Swiss German people also master standard (or ’High’) Ger- man which is taught in school as the official language of communication. Swiss German is varies strongly. Many differences exist in the dialectal continuum of the German speaking part of Switzerland. Besides pronunciation, it also varies a lot in writing. Standard German used to be the exclusive lan- guage for writing in Switzerland. Writing in Swiss German has only come up rather recently (notably in text messag- ing). Because of this, there are no orthographic conventions for Swiss German varieties. Even people speaking the same dialect can, and often do, write phonetically identical words differently. In this paper, we present a dictionary of written standard German words paired with their pronunciation in Swiss German words. Additionally Swiss German spontaneous writings, i.e. writings as they may be used in text messages by native speakers, are paired with Swiss German pronun- ciations. The primary motivation for building this dictionary is ren- dering Swiss German accessible for technologies such as Automatic Speech Recognition (ASR). This is the first publicly described Swiss German dictionary shared for research purposes. Furthermore, this is the first dictionary that combines pronunciations …

Content truncated to 500 words