The German Reference Corpus DeReKo: New Developments – New Opportunities

The German Reference Corpus DeReKo: New Developments – New Opportunities Marc Kupietz∗, Harald Lüngen∗, Paweł Kamocki∗†, Andreas Witt∗ ∗Institut für Deutsche Sprache, R5, 6-13; D-68161 Mannheim †European Language Resources Distribution Agency 9 rue des Cordelières; F-75013 Paris {kupietz|luengen|witt}@ids-mannheim.de, kamocki@elda.org Abstract This paper discusses current trends in DeReKo, the German Reference Corpus, concerning legal issues around the recent German copyright reform with positive implications for corpus building and corpus linguistics in general, recent corpus extensions in the genres of popular magazines, journals, historical texts, and web-based football reports. Besides, DeReKo is finally accessible via the new corpus research platform KorAP, offering registered users several news features in comparison with its predecessor COSMAS II. Keywords: reference corpus, very large corpus, diversity, intellectual property rights, copyright reform, corpus analysis, compa- rable corpora, collocation analysis, word embeddings 1. Introduction The German Reference Corpus DeReKo is presumably the largest archive of German language texts designed for lin- guistic research (Kupietz et al., 2010). As of 2018 (Institut für Deutsche Sprache, 2018), it contains more than 42 bil- lion tokens, comprising a multitude of genres such as news- paper text, fiction, or specialised text, with a current growth rate of 3.1 billion word per year. Besides the constant ac- quisition of new newspaper sources, one focus of corpus extension in the past four years has been on the curation of content from sources of computer-mediated communi- cation (CMC), cf. Margaretha and Lüngen (2014); Schröck and Lüngen (2015); Lüngen et al. (2016). In 2017, several other new genres that had previously not been available in DeReKo, have been acquired and included in the latest release (Institut für Deutsche Sprache, 2018). These are discussed in Section 3. Likewise as of 2017, the bulk of DeReKo can be accessed online via the new corpus research platform KorAP, which offers several new corpus query features, which are discussed in Section 4. Actual and envisioned improvements in DeReKo’s legal status are discussed in Section 2. 2. Legal and Licensing Situation 2.1. License Types The Institut für Deutsche Sprache (IDS), DeReKo’s host in- stitution, is not the owner of DeReKo’s content. Rather, we have more than 200 license agreements with rights holders, mostly publishers, granting the use of texts for non-commercial, scientific research by registered users and strictly within the query-and-analysis-only framework (cf. Kupietz/Lüngen, 2014) offered through the corpus research interfaces of COSMAS II and KorAP (see Section 4.). Due to the fact that the range of lexicographically edited dictionaries and lexicons on the German language offered by commercial publishers has deteriorated considerably in the last few years, IDS is interested in acquiring addi- tional licensing that would allow for building, editing, and (semi-)commercially publishing fundamentals for compre- hensive print and on-line dictionaries based on statistical and lexicographic analyses and evaluations of (parts of) DeReKo, in the future. Such a license extension towards commercial use could also open new application fields e. g. for DeReKo-based language models in commercial contexts and also in academic contexts whenever copies of DeReKo(- parts) are required …

Content truncated to 500 words