Detecting Troll Tweets in a Bilingual Corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6247–6254 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6247 Detecting Troll Tweets in a Bilingual Corpus Lin Miao1, Mark Last1, Marina Litvak2 Ben-Gurion University of the Negev1, Shamoon College of Engineering2 P.O.B. 653 Beer-Sheva 8410501 Israel1, 56 Bialik St. Be’er Sheva 8410802 Israel2 miaol@post.bgu.ac.il, mlast@bgu.ac.il, litvak.marina@gmail.com Abstract During the past several years, a large amount of troll accounts has emerged with efforts to manipulate public opinion on social network sites. They are often involved in spreading misinformation, fake news, and propaganda with the intent of distracting and sowing discord. This paper aims to detect troll tweets in both English and Russian assuming that the tweets are generated by some “troll farm." We reduce this task to the authorship verification problem of determining whether a single tweet is authored by a “troll farm" account or not. We evaluate a supervised classification approach with monolingual, cross-lingual, and bilingual training scenarios, using several machine learning algorithms, including deep learning. The best results are attained by the bilingual learning, showing the area under the ROC curve (AUC) of 0.875 and 0.828, for tweet classification in English and Russian test sets, respectively. It is noteworthy that these results are obtained using only raw text features, which do not require manual feature engineering efforts. In this paper, we introduce a resource of English and Russian troll tweets containing original tweets and translation from English to Russian, Russian to English. It is available for academic purposes. Keywords: Bilingual Training, Authorship Verification, Social Network Analysis, Natural Language Processing 1. Introduction With the rise of social media, people are exposed to large amounts of information on social media platforms, which creates the opportunity for some organizations to distribute rumors, misinformation, and speculation, in an attempt to manipulate the opinion of the public. A Kremlin-linked propaganda organization, known as the Internet Research Agency (IRA), has been assessed by the U.S. Intelligence Community to be part of a Russian state-run effort to in- fluence the outcome of the 2016 U.S. presidential race1. As a result, Twitter suspended the IRA-connected accounts and deleted 200,000 Russian troll tweets. Twitter has been flooded with false news, and propaganda-spreading trolls. This dedicates the importance of detecting troll tweets to protect the public from the inappropriate content in the so- cial network. Considering that there are many languages in Twitter data, the detection of troll tweets needs to be able to handle mul- tiple languages, to cover a larger portion of available troll tweets. The dataset2 we use is constructed by the tweets from IRA-related troll accounts. Because English and Rus- sian represent the largest percentages of this dataset, we will handle English and Russian tweets to cover a larger portion of available troll tweets. The aim of this work is to determine whether a given tweet is a troll tweet, based solely on its content. The task of computational detection of troll tweets in a multilingual …

Content truncated to 500 words