Verifiably Effective Arabic Dialect Identification

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1465–1468, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Verifiably Effective Arabic Dialect Identification Kareem Darwish, Hassan Sajjad, Hamdy Mubarak Qatar Computing Research Institute Qatar Foundation {kdarwish,hsajjad,hmubarak}@qf.org.qa Abstract Several recent papers on Arabic dialect identi- fication have hinted that using a word unigram model is sufficient and effective for the task. However, most previous work was done on a standard fairly homogeneous dataset of dialec- tal user comments. In this paper, we show that training on the standard dataset does not generalize, because a unigram model may be tuned to topics in the comments and does not capture the distinguishing features of dialects. We show that effective dialect identification requires that we account for the distinguishing lexical, morphological, and phonological phe- nomena of dialects. We show that accounting for such can improve dialect detection accu- racy by nearly 10% absolute. 1 Introduction Modern Standard Arabic (MSA) is the lingua franca of the so-called Arab world, which includes north- ern Africa, the Arabian Peninsula, and Mesopotamia. However, Arabic speakers generally use dramatically different languages (or dialects) in daily interactions and in social media. These dialects may differ in vocab- ulary, morphology, and spelling from MSA and most do not have standard spellings. There is often large lexical overlap between dialects and MSA. Performing proper Arabic dialect identification may positively im- pact many Natural Language Processing (NLP) appli- cation. For example, transcribing dialectal speech or automatically translating into a particular dialect would be aided by the use of targeted language models that are trained on texts in that dialect. This has led to recent interest in automatic identifi- cation of different Arabic dialects (Elfardy et al., 2013; Cotterell et al., 2014; Zaidan et al., 2014). Though pre- vious work (Cotterell et al., 2014) have reported high accuracies for dialect identification using word uni- gram model, which implies that this is a solved prob- lem, we argue that the problem is far from being solved. The reason for this assertion stems from the fact that the available dialectal data is drawn from singular sources, namely online news sites, for each dialect. This is prob- lematic because comments on singular news site are likely to have some homogeneity in topics and jargon. Such homogeneity has caused fairly simple classifica- tion techniques that use word unigrams and character n- grams to yield very high identification accuracies. Per- haps, this can be attributed to topical similarity and not just differences between dialects. To showcase this, we trained a classifier using the best reported methods, and we tested the classifier on a new test set of 700 tweets, with dialectal Egyptian (ARZ) and MSA tweets, which led to a low accuracy of 83.3%. We also sorted words in the ARZ part from our training dataset by how much they discriminate between ARZ and MSA (using mu- tual information) and indeed many of the top words were in fact MSA words. There …

Content truncated to 500 words