#Turki$hTweets: A Benchmark Dataset for Turkish Text Correction

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4190–4198 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4190 #Turki$hTweets: A Benchmark Dataset for Turkish Text Correction Asiye Tuba Koksal1, Ozge Bozal1,2, Emre Yurekli1, Gizem Gezici1,3 1Huawei R&D Center, Istanbul, Turkey 2Bogazici University, Istanbul, Turkey 3Sabanci University, Istanbul, Turkey {asiye.tuba.koksal, ozge.bozal, emre.yurekli, gizem.gezici}@huawei.com Abstract #Turki$hTweets is a benchmark dataset for the task of correcting the user misspellings, with the purpose of introducing the first public Turkish dataset in this area. #Turki$hTweets provides correct/incorrect word annotations with a detailed misspelling category formula- tion based on the real user data. We evaluated four state-of-the-art approaches on our dataset to present a preliminary analysis for the sake of reproducibility. The annotated dataset is publicly available at https://github.com/ atubakoksal/annotated_tweets. 1 Introduction The extensive use of social media platforms such as Twitter, Facebook, forums, and blogs has cre- ated a new form of written language, which is full of intentional and unintentional misspellings as well as newly-emerged slang words and abbrevia- tions. This new type of language poses significant challenges for various natural language process- ing (NLP) tasks, mostly requiring properly written textual content for analysis. Therefore, text nor- malization, i.e., transforming non-standard words into their standard forms and spelling correction, i.e., correcting unintentional spelling errors, have become indispensable pre-processing stages. The pre-processing phase is known to boost the model performance for the various NLP applications, in- cluding but not limited to POS tagging, sentiment classification and search1. Although correcting misspelling errors is crucial for NLP applications, it is generally not straight- forward and even challenging for morphologically rich languages. There exist many different surface forms of a single word in highly agglutinative lan- guages. Specifically for Turkish, suffixes should 1Query correction can help to increase the search perfor- mance by correctly understanding user intent. also comply with the vowels and the last letter of the word. This leads to many different variations of a single word, thereby increasing the possibility of misspelling errors. Previous studies adapted for English do not fit and thus, there is the need for resources tailored particularly for these languages. Currently, there is no publicly available dataset in this area that can be used for model evaluation with reproducible results. In the online platforms, there exist not only in- tentionally misspelled words but also unintentional spelling errors, both constituting out-of-vocabulary (OOV) words in the textual content. Intentional user misspellings are quite frequent, particularly in online media and these errors highly vary de- pending on the intention of use. Therefore, a more fine-grained analysis of the error types is required by categorizing the errors and further including them in the evaluation dataset for a proper model evaluation. In this way, a particular model can be assessed whether it provides generalizable results for the corresponding language. Based on these, we introduce a new Turkish dataset by categorizing, annotating and correcting the distinct misspelling types in text. Moreover, we make a fine-grained evaluation of the selected state- of-the-art approaches in …

Content truncated to 500 words