DART: A Large Dataset of Dialectal Arabic Tweets

DART: A Large Dataset of Dialectal Arabic Tweets Israa Alsarsour, Esraa Mohamed, Reem Suwaileh, Tamer Elsayed Computer Science and Engineering Department, Qatar University, Doha, Qatar {ia1205702, em1205267, reem.suwaileh, telsayed}@qu.edu.qa Abstract In this paper, we present a new large manually-annotated multi-dialect dataset of Arabic tweets that is publicly available. The Dialectal ARabic Tweets (DART) dataset has about 25K tweets that are annotated via crowdsourcing and it is well-balanced over five main groups of Arabic dialects: Egyptian, Maghrebi, Levantine, Gulf, and Iraqi. The paper outlines the pipeline of constructing the dataset from crawling tweets that match a list of dialect phrases to annotating the tweets by the crowd. We also touch some challenges that we face during the process. We evaluate the quality of the dataset from two perspectives: the inter-annotator agreement and the accuracy of the final labels. Results show that both measures were substantially high for the Egyptian, Gulf, and Levantine dialect groups, but lower for the Iraqi and Maghrebi dialects, which indicates the difficulty of identifying those two dialects manually and hence automatically. Keywords: Arabic, Multi-Dialect, Twitter, Crowdsourcing, Annotations, Corpus 1. Introduction The Arabic language is the fifth most widely spoken lan- guage in the world; more than 380 million people speak and write in Arabic(Darwish et al., 2014). Additionally, ap- proximately 41.7% of Arabic speakers are using the Inter- net1 which necessitates the need for developing language- specific tools for Arabic. The Arabic language has many dialects (varieties), besides the Modern standard Arabic (MSA), that are broadly used in daily life(Huang, 2015). Al- though dialects have common linguistic uses, they greatly differ making Arab people themselves face difficulty in understanding each other. The variations and similarities of Arabic dialects stem from different factors, e.g., social class, education level, religion, gender, and geographical re- gion (Benajiba and Diab, 2010). Arab users write in dialects over the Internet and exten- sively in social media. This introduces many challenges to researchers in areas such as Natural Language Program- ming, Information Retrieval, and Machine Learning, who deal with the spoken and/or written language one way or the other. The advancement in these areas is remarkably restricted by the shortage of high-quality Arabic language resources. In this paper, we tackle the problem of building a large di- alectal Arabic tweets dataset that somewhat remedies the lack of Arabic resources and opens the door of support to tackle various research problems such as dialect detection, words segmentation, translation, cross-dialect search, and speech recognition. Our contribution in this work is two folds: 1. We introduce DART dataset; a large well-balanced publicly-available2 Dialectal ARabic Tweets dataset that we believe will enable research on different areas. 2. We provide an analysis on the quality of the dataset in terms of inter-annotator agreement (measured in Kappa), and accuracy of final labels (measured by in- house annotators). 1http://www.internetworldstats.com/stats19.htm 2http://qufaculty.qu.edu.qa/telsayed/datasets/ The rest of this paper is organized as follows. Section 2 reviews the literature and describes the publicly-available datasets. We layout the process of collecting and annotating DART dataset in …

Content truncated to 500 words