Building a User-Generated Content North-African Arabizi Treebank: Tackling Hell

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1139–1150 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1139 Building a User-Generated Content North-African Arabizi Treebank: Tackling Hell Djamé Seddah1 Farah Essaidi1 Amal Fethi1 Matthieu Futeral1 Benjamin Muller1,2 Pedro Javier Ortiz Suárez1,2 Benoît Sagot1 Abhishek Srivastava1 1Inria, Paris, France 2Sorbonne Université, Paris, France firstname.lastname@inria.fr Abstract We introduce the first treebank for a romanized user-generated content variety of Algerian, a North-African Arabic dialect known for its fre- quent usage of code-switching. Made of 1500 sentences, fully annotated in morpho-syntax and Universal Dependency syntax, with full translation at both the word and the sentence levels, this treebank is made freely available. It is supplemented with 50k unlabeled sen- tences collected from Common Crawl and web- crawled data using intensive data-mining tech- niques. Preliminary experiments demonstrate its usefulness for POS tagging and dependency parsing. We believe that what we present in this paper is useful beyond the low-resource language community. This is the first time that enough unlabeled and annotated data is pro- vided for an emerging user-generated content dialectal language with rich morphology and code switching, making it an challenging test- bed for most recent NLP approaches. 1 Introduction Until the rise of fully unsupervised techniques that would free our field from its addiction to anno- tated data, the question of building useful data sets for under-resourced languages at a reasonable cost is still crucial. Whether the lack of labeled data originates from being a minority language status, its almost oral-only nature or simply its programmed political disappearance, geopolitical events are a factor highlighting a language defi- ciency in terms of natural language processing re- sources that can have an important societal impact. Events such as the Haïti crisis in 2010 (Munro, 2010) and the current Algerian revolts (Nossiter, 2019)1 are massively reflected on social media, yet often in languages or dialects that are poorly re- 1https://www.nytimes.com/2019/03/01/world/ africa/algeria-protests-bouteflika.html sourced, namely Haitian Creole and Algerian di- alectal Arabic in these cases. No readily avail- able parsing and machine translations systems are available for such languages. Taking as an ex- ample the Arabic dialects spoken in North-Africa, mostly from Morocco to Tunisia, sometimes called Maghribi, sometimes Darija, these idioms notori- ously contain various degrees of code-switching with languages of former colonial powers such as French, Spanish, and, to a much lesser extent, Ital- ian, depending on the area of usage (Habash, 2010; Cotterell et al., 2014; Saadane and Habash, 2015). They share Modern Standard Arabic (MSA) as their matrix language (Myers-Scotton, 1993), and of course present a rich morphology. In conjunc- tion with the resource scarcity issue, the code- switching variability displayed by these languages challenges most standard NLP pipelines, if not all. What makes these dialects especially interesting is their widespread use in user-generated content found on social media platforms, where they are generally written using a romanized version of the Arabic script, called Arabizi, which is neither stan- dardized nor formalized. The absence of standard- ization …

Content truncated to 500 words