Turkish Tweet Classification with Transformer Encoder

Proceedings of Recent Advances in Natural Language Processing, pages 1380–1387, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_158 1380 Turkish Tweet Classification with Transformer Encoder Atıf Emre Y¨uksel∗ Bo˘gazic¸i University ˙Istanbul, Turkey Yas¸ar Alim T¨urkmen∗ Bo˘gazic¸i University ˙Istanbul, Turkey { atif.yuksel, yasar.turkmen, arzucan.ozgur }@boun.edu.tr Arzucan ¨Ozg¨ur Bo˘gazic¸i University ˙Istanbul, Turkey Ays¸e Berna Altınel Marmara University ˙Istanbul, Turkey berna.altinel@marmara.edu.tr Abstract Short-text classification is a challenging task, due to the sparsity and high dimen- sionality of the feature space. In this study, we aim to analyze and classify Turkish tweets based on their topics. Social me- dia jargon and the agglutinative structure of the Turkish language makes this clas- sification task even harder. As far as we know, this is the first study that uses a Transformer Encoder for short text classi- fication in Turkish. The model is trained in a weakly supervised manner, where the training data set has been labeled automat- ically. Our results on the test set, which has been manually labeled, show that per- forming morphological analysis improves the classification performance of the tra- ditional machine learning algorithms Ran- dom Forest, Naive Bayes, and Support Vector Machines. Still, the proposed ap- proach achieves an F-score of 89.3% out- performing those algorithms by at least 5 points. 1 Introduction Short-text usage is increasing day by day and we encounter short-text messages on many social me- dia platforms in different forms such as tweets, Facebook status posts, or microblog entries. Twit- ter is one of the most widely used platforms, where a huge amount of short-texts are produced. More than 500 million tweets are posted on a typical day (Aslam, 2018). People use Twitter in order to pro- duce and reach information faster about the topic they are interested in. Therefore, tweet classifica- tion becomes an important task to improve tweet filtering and tweet recommendation. ∗Equal contribution Traditional machine learning algorithms such as K-Nearest Neighbor, Support Vector Machines (SVM), and Naive Bayes have been widely used for text classification tasks and accuracy levels of over 80% have been reported in the literature for various data sets (Kadhim, 2019). However, ob- taining a similar level of success for short-text classification is difficult, since short-texts contain smaller number of words compared to lengthy texts, which makes classifying them effectively a challenging task (Taksa et al., 2007). While a number of studies have been conducted for short- text classification, most of them have addressed the task of English tweet classification (Batool et al., 2013; Selvaperumal & Suruliandi, 2014). In this paper, we tackle the task of Turkish tweet classification. The grammatical and syntactic fea- tures of the Turkish language pose additional chal- lenges for short-text classification. The aggluti- native nature of Turkish results in a high num- ber of different word surface forms, since a root word can take many different derivational and in- flectional affixes. This leads to the data sparse- ness problem. We propose a Transformer-Encoder based model for Turkish topic-based tweet clas- sification and compare it with the traditional ma- chine learning …

Content truncated to 500 words