Fine-Grained Arabic Dialect Identification

Proceedings of the 27th International Conference on Computational Linguistics, pages 1332–1344 Santa Fe, New Mexico, USA, August 20-26, 2018. 1332 Fine-Grained Arabic Dialect Identification Mohammad Salameh† Houda Bouamor† Nizar Habash‡ †Carnegie Mellon University in Qatar ‡New York University Abu Dhabi {msalameh,hbouamor}@qatar.cmu.edu nizar.habash@nyu.edu Abstract Previous work on the problem of Arabic Dialect Identification typically targeted coarse-grained five dialect classes plus Standard Arabic (6-way classification). This paper presents the first results on a fine-grained dialect classification task covering 25 specific cities from across the Arab World, in addition to Standard Arabic – a very challenging task. We build several classification systems and explore a large space of features. Our results show that we can identify the exact city of a speaker at an accuracy of 67.9% for sentences with an average length of 7 words (a 9% relative error reduction over the state-of-the-art technique for Arabic dialect identification) and reach more than 90% when we consider 16 words. We also report on additional insights from a data analysis of similarity and difference across Arabic dialects. Title and Abstract in Arabic ���������� ����� ����� �������� �� �� ��������� ����������� ��� ����� ����� ��� ��� ���������� ��������� ��� ������ ��� ��������� ��� ���������� ����� ���� �������� ������� �� �� ���������� ������� ��� �������� ������������ ������� ������� � ��� ������ � �������� ����������� ���� �� �������� ���������� ������ ����� ���� ��� ���������� ��� ���� ����� �� ���������� ����� ���� ��� �������� ������ �������� ������ ��� �������� �� �� ���� ��������� ������������ ��� �������� ����� ���� �������� �� �� ������ ��� �������� � �� �� ��� ����� ���� ������ ����������� ��� ������ ���� �� ������ � �������� � ��� �� �� � ������� ��������� ��� ���� ��� �������� ������� ��������� ���������� ���� �� �������� ����������� ����� ������� �������� ������ ������������ �� ���� �������� ��� ���� � �������� ���������� ���������� ��� ������ � ����� ���� ��� � ����� �� �������� �� �� ��� � �� �������� � ������ � ���� ��� �� ������� �������� ��� �� �� ��� � ��,� � ���������� ����� ���� ������ ����� ����� ���� ������ ��� ������� ������� �������� ��� �������� ��������� ������� 1 Introduction Dialect identification (DID) is the task of automatically identifying the dialect of a particular segment of speech or text of any size (i.e., word, sentence, or document). This task has attracted increasing attention in recent years. For instance, several evaluation campaigns were dedicated to discriminating between language varieties (Malmasi et al., 2016; Zampieri et al., 2017). This is not surprising considering the importance of automatic DID for several NLP tasks, where prior knowledge about the dialect of an input text can be helpful, such as machine translation (Salloum et al., 2014), sentiment analysis (Al-Twairesh et al., 2016), or author profiling (Sadat et al., 2014). For Arabic DID, previous work typically targeted coarse-grained five dialect classes plus Standard Arabic at most (6-way classification) (Zaidan and Callison-Burch, 2014; Elfardy and Diab, 2013; Dar- wish et al., 2014). In this paper, we tackle a finer-grained dialect classification task, covering 25 cities …

Content truncated to 500 words