Detecting de minimis Code-Switching in Historical German Books

Proceedings of the 28th International Conference on Computational Linguistics, pages 1808–1814 Barcelona, Spain (Online), December 8-13, 2020 1808 Detecting de minimis Code-Switching in Historical German Books Shijia Liu David Smith Khoury College of Computer Sciences Northeastern University liu.shij@northeastern.edu, dasmith@ccs.neu.edu Abstract Code-switching has long interested linguists, with computational work in particular focusing on speech and social media data (Sitaram et al., 2019). This paper contrasts these informal instances of code-switching to its appearance in more formal registers, by examining the mixture of languages in the Deutsches Textarchiv (DTA), a corpus of 1406 primarily German books from the 17th to 19th centuries. We automatically annotate and manually inspect spans of six embedded languages (Latin, French, English, Italian, Spanish, and Greek) in the corpus. We quantitatively analyze the differences between code-switching patterns in these books and those in more typically studied speech and social media corpora. Furthermore, we address the practical task of predicting code-switching from features of the matrix language alone in the DTA corpus. Such classifiers can help reduce errors when optical character recognition or speech transcription is applied to a large corpus with rare embedded languages. 1 Introduction Code-switching, the linguistic phenomenon where speakers or writers alternate between different lan- guages in a single utterance or statement, is commonly seen in bilingual or multilingual communities. In the last few decades, code-switching has drawn scholarly attention in computational linguistics and natu- ral language processing from many different perspectives (Sitaram et al., 2019). Researchers from formal linguistics, psycholinguistics, sociolinguistics, philosophy, anthropology, and elsewhere have considered the phenomenon (Nilep, 2006). In formal linguistics, interest has risen in studying syntactic and mor- phosyntactic constraints on language alternation (Nilep, 2006). In sociolinguistics, there has been a focus on the social context in conversations where code-switching happens. For example, Blom and Gumperz (1972) proposes the dichotomy of situational and metaphorical code-switching, which serve as indicators of whether different languages or language varieties are used in different social situations. In natural language processing, there is extensive work studying code-switching from an engineering perspective. Different NLP tasks have been proposed on code-switching corpora, such as language ID (Solorio et al., 2014; Sequiera et al., 2015), named entity recognition (Aguilar et al., 2018; Singh et al., 2018) , POS tagging (Solorio and Liu, 2008; Vyas et al., 2014), sentiment analysis (Vilares et al., 2015), automatic speech recognition (Chan et al., 2014; Weiner et al., 2012), question answering (Chandu et al., 2018), etc. Indeed, with the abundance of code-switched speech and (informal) text data, there are many choices of NLP directions that one can pursue. Code-switching, as a language phenomenon, is usually considered informal. It is often found in speech and in casual text, such as social media (Sitaram et al., 2019); however, code-switching also appears in formal settings, such as newspaper reports, or in this paper, books. Table 1 shows examples from the Deutsches Textarchiv (DTA) corpus of code-switching from German into Latin, English, French, and Greek. Based on such observations, we ask: from a quantitative perspective, precisely how …

Content truncated to 500 words