A Twitter Corpus for Named Entity Recognition in Turkish

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4546–4551 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4546 A Twitter Corpus for Named Entity Recognition in Turkish Buse C¸ arık, Reyyan Yeniterzi Sabanci University Istanbul, Turkey {busecarik, reyyan}@sabanciuniv.edu Abstract This paper introduces a new Turkish Twitter Named Entity Recognition dataset. The dataset, which consists of 5000 tweets from a year-long period, was labeled by multiple annotators with a high agreement score. The dataset is also diverse in terms of the named entity types as it contains not only person, organization, and location but also time, money, product, and tv-show categories. Our initial experiments with pretrained language models (like BertTurk) over this dataset returned F1 scores of around 80%. We share this dataset publicly. Keywords: Twitter, Named Entity Recognition, Turkish 1. Introduction Named Entity Recognition (NER), a subtask of infor- mation extraction is used to identify predefined named entities (NEs) such as temporal and numerical ex- pressions alongside person, location, or organization names. Researchers have achieved outstanding results in well-studied languages such as English for the NER task, which has been used as part of several other NLP tasks such as summarization, question answering, and entity linking. However, it remains a subject of study for languages that lack sufficient research and re- sources, such as Turkish. There are several reasons for this. First of all, the majority of previous studies in Turkish NER focused on formal writings that comply with grammatical and spelling rules (T¨ur et al., 2003; Tatar and Cicekli, 2011). In limited studies to date (C¸ elikkaya et al., 2013; Okur et al., 2018), the application of the developed NER models to informal texts such as mini-blogs has yielded poor results. Nevertheless, with the growth in the amount of social media content, the need to recog- nize NEs in these noisy texts has increased. In addition, apart from the NE types defined by the Message Understanding Conference (MUC) series (Gr- ishman and Sundheim, 1996), new NE types have not been adequately studied for Turkish except for a few studies (K¨uc¸¨uk et al., 2014). However, investigating different types of NEs helps improving the results when these NE types are tailored to other NLP tasks. Another significant issue in Turkish NER studies is that most of the datasets, especially the informal ones, are not publicly available. Only K¨uc¸¨uk et al. (2014) and K¨uc¸¨uk and Can (2019) released their datasets of tweets publicly by providing the tweet IDs. However, these two datasets are limited both in terms of diversity and size. In this work, we introduce a new dataset for Turk- ish NER gathered from Twitter distributed uniformly across months over a long time. We included under- studied NE types in our label set and obtained a high agreement score among multiple annotators. We also present some initial results on this dataset. Since transformer-based models outperform in many NLP tasks, we experimented with different variations of these models on our dataset as …

Content truncated to 500 words