A Gold Standard Dependency Treebank for Turkish

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5156–5163 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5156 A Gold Standard Dependency Treebank for Turkish Tolga Kayadelen†, Adnan ¨Ozt¨urel†, Bernd Bohnet‡ Google †London, UK ‡Amsterdam, Netherlands {tkayadelen, ozturel, bohnetbd}@google.com Abstract We introduce TWT; a new treebank for Turkish which consists of web and Wikipedia sentences that are annotated for segmentation, morphology, part-of-speech and dependency relations. To date, it is the largest publicly available human-annotated morpho-syntactic Turkish treebank in terms of the annotated word count. It is also the first large Turkish dependency treebank that has a dedicated Wikipedia section. We present the tagsets and the methodology that are used in annotating the treebank and also the results of the baseline experiments on Turkish dependency parsing with this treebank. Keywords: treebank, dependency parsing, Turkish 1. Introduction Dependency parsing is an important building block in im- proving the performance of downstream NLP tasks such as semantic role labeling (Marcheggiani et al., 2017), rela- tion extraction (Zhang et al., 2018) or machine translation (Chen et al., 2017). Treebanks are invaluable resources for developing and training accurate dependency parsers in su- pervised settings and more and more research has been in- vested in developing high quality treebanks for various lan- guages over the years. Compared to other well studied languages in NLP, the pub- licly available treebanks for Turkish has remained meagre in size and domain variation until very recently. The METU Sabancı Treebank (MST) (Atalay et al., 2003; Oflazer et al., 2003) had been the only treebank for Turkish for over a decade. Even though the cumulative size of morpho- syntactically annotated datasets has increased with the re- lease of new language resources such as ITU Web Tree- bank (IWT), (Pamay et al., 2015) and Parallel Universal Dependencies Treebank (PUD) (Zeman et al., 2018), Turk- ish continues to stay relatively under resourced as the num- ber of annotated sentences in all of these above-mentioned language resources amount to approximately 12K unique sentences in total. Together with this paper we open-source the Turkish Web Treebank (TWT)1, which is freely available, large in size, and also involves annotated sentences from previously un- der represented domains such as Wikipedia. We present our part-of-speech (PoS) and morphological feature tagsets and dependency label set, our annotation methodology, inter- annotator agreement scores, and discuss some of the high- lights of our annotation decisions – especially the cases where they diverge from the previously implemented an- notation practices of the Turkish treebanking literature. We also show that the tagset used in the annotation of TWT is the most elaborate one that has been developed so far for Turkish. It is adequate in representing the complex deriva- tional morphology of Turkish throughout part-of-speech 1https://github.com/ google-research-datasets/turkish-treebanks/ and dependency labeling. We illustrate that this level of representational granularity does not come at the expense of parsing performance. 2. A Brief History of Turkish Treebanking Turkish treebanking dates back to MST (Atalay et al., …

Content truncated to 500 words