Domain Adaptation for Arabic Cross-Domain and Cross-Dialect Sentiment Analysis from Contextualized Word Embedding

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2824–2837 June 6–11, 2021. ©2021 Association for Computational Linguistics 2824 Domain Adaptation for Arabic Cross-Domain and Cross-Dialect Sentiment Analysis from Contextualized Word Embedding Abdellah El Mekki1 Abdelkader El Mahdaouy1 Ismail Berrada1 Ahmed Khoumsi2 1School of Computer Sciences, Mohammed VI Polytechnic University, Morocco 2Dept. Electrical & Computer Engineering, University of Sherbrooke, Canada {firstname.lastname}@um6p.ma ahmed.khoumsi@usherbrooke.ca Abstract Finetuning deep pre-trained language models has shown state-of-the-art performances on a wide range of Natural Language Processing (NLP) applications. Nevertheless, their gen- eralization performance drops under domain shift. In the case of Arabic language, diglos- sia makes building and annotating corpora for each dialect and/or domain a more challenging task. Unsupervised Domain Adaptation tack- les this issue by transferring the learned knowl- edge from labeled source domain data to unla- beled target domain data. In this paper, we pro- pose a new unsupervised domain adaptation method for Arabic cross-domain and cross- dialect sentiment analysis from Contextual- ized Word Embedding. Several experiments are performed adopting the coarse-grained and the fine-grained taxonomies of Arabic dialects. The obtained results show that our method yields very promising results and outperforms several domain adaptation methods for most of the evaluated datasets. On average, our method increases the performance by an im- provement rate of 20.8% over the zero-shot transfer learning from BERT. 1 Introduction The Arabic language is characterized by two main language varieties: Modern Standard Ara- bic (MSA) and Arabic dialect. MSA has a stan- dard written form and acquires an official status across the Arab countries, while Dialectal Ara- bic refers to the informal spoken dialects in the Arab World (Habash, 2010). These dialects are used in daily life but have no standard written form (Saadane and Habash, 2015; Habash et al., 2018; Eryani et al., 2020). Geographically and accord- ing to (Zaidan and Callison-Burch, 2014), Arabic dialects can be classified into five coarse-grained regional dialects: Egyptian, Levantine, Gulf, Iraqi, and Maghrebi. Recent studies have categorized dialectal Arabic into more fine-grained levels, in- cluding countries and cities (Bouamor et al., 2019; Muhammad et al., 2020). These dialects differ from one another and from MSA, to a varying degree, at different linguistic levels (Salameh et al., 2018; Erdmann et al., 2018). With the unprecedented reach of social media platforms, Sentiment Analysis (SA) has become a fundamental task for many applications. Most research work in this area has been devoted to En- glish and other European languages, while some research studies have addressed the question of transfer learning from MSA to dialectal Arabic. However, Khaddaj et al. (2019) and Qwaider et al. (2019) have shown that zero-shot transfer learning, from models trained on MSA data, does not per- form well for SA on dialectal Arabic data. So, exist- ing works have focused on building resources and annotating corpora for a few dialects where most of them were collected from social media (Med- haffar et al., 2017; Al-Twairesh et al., 2017; Baly …

Content truncated to 500 words