SYN2015: Representative Corpus of Contemporary Written Czech

SYN2015: Representative Corpus of Contemporary Written Czech Michal Křen, Václav Cvrček, Tomáš Čapka, Anna Čermáková, Milena Hnátková, Lucie Chlumská, Tomáš Jelínek, Dominika Kováříková, Vladimír Petkevič, Pavel Procházka, Hana Skoumalová, Michal Škrabal, Petr Truneček, Pavel Vondřička, Adrian Jan Zasina Charles University in Prague, Faculty of Arts Nám. Jana Palacha 2, 116 38 Praha 1, Czech Republic {michal.kren, vaclav.cvrcek, tomas.capka, anna.cermakova, milena.hnatkova, lucie.chlumska, tomas.jelinek, dominika.kovarikova, vladimir.petkevic, pavel.prochazka, hana.skoumalova, michal.skrabal, petr.trunecek, pavel.vondricka, adrian.zasina}@ff.cuni.cz Abstract The paper concentrates on the design, composition and annotation of SYN2015, a new 100-million representative corpus of contemporary written Czech. SYN2015 is a sequel of the representative corpora of the SYN series that can be described as traditional (as opposed to the web-crawled corpora), featuring cleared copyright issues, well-defined composition, reliability of annotation and high-quality text processing. At the same time, SYN2015 is designed as a reflection of the variety of written Czech text production with necessary methodological and technological enhancements that include a detailed bibliographic annotation and text classification based on an updated scheme. The corpus has been produced using a completely rebuilt text processing toolchain called SynKorp. SYN2015 is lemmatized, morphologically and syntactically annotated with state-of-the-art tools. It has been published within the framework of the Czech National Corpus and it is available via the standard corpus query interface KonText at http://kontext.korpus.cz as well as a dataset in shuffled format. Keywords: Czech, language corpus, representativeness size (# of running words) type publication year SYN2000 100 mil. general, representative mostly 1990–1999 SYN2005 100 mil. general, representative mostly 2000–2004 SYN2006PUB 300 mil. newspapers and magazines 1989–2004 SYN2009PUB 700 mil. newspapers and magazines 1995–2007 SYN2010 100 mil. general, representative mostly 2005–2009 SYN2013PUB 935 mil. newspapers and magazines 2005–2009 SYN2015 100 mil. general, representative mostly 2010–2014 Table 1: The SYN-series corpora of contemporary written Czech. 1. Background The Czech National Corpus (CNC) aims at extensive and continuous mapping of the Czech language in the whole spectrum of its varieties and forms. This effort results in compilation, maintenance and providing access to a number of corpora (synchronic/diachronic, written/spoken, monolingual/parallel etc.), including corpora of contemporary written Czech making up the SYN series. The SYN-series corpora can be described as traditional (as opposed to the web-crawled corpora), featuring cleared copyright issues, well-defined composition, reliability of annotation and high-quality text processing (Hnátková et al., 2014). 2. Representative Corpora of the SYN Series Currently, the SYN series consists of four 100-million representative corpora of written Czech (SYN2000, SYN2005, SYN2010, and SYN2015; the number denotes the corpus publication year) and three large newspaper corpora (SYN2006PUB, SYN2009PUB, SYN2013PUB). As all the corpora are disjoint, i.e. any document can be included only into one of them, their total size exceeds 2 billion tokens. The representative SYN-series corpora cover four consecutive time periods in a regular five-year interval (with the exception of SYN2000 which covers the period of ten years, i.e. 1990–1999) and they contain a large 2522 variety of written genres in proportions based on language reception studies (Králík & Šulc, 2005). Their design, comparability, strengths and weaknesses are …

Content truncated to 500 words