Standardisation of Dialect Comments in Social Networks in View of Sentiment Analysis : Case of Tunisian Dialect

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 5436–5443 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 5436 Standardization of Dialect Comments in Social Networks in View of Sentiment Analysis : Case of Tunisian Dialect Sam´eh Kchaou, Rahma Boujelbane, Emna FsiH, Lamia Hadrich Belguith ANLP Research group, MIRACL Lab. FSEGS, University of Sfax, Tunisia samehkchaou4@gmail.com, rahma.boujelban@gmail.com, emnafsih@gmail.com, lamia.belguith@fsegs.usf.tn Abstract With the growing access to the internet, the spoken Arabic dialect language becomes an informal languages written in social media. Most users post comments using their own dialect. This linguistic situation inhibits mutual understanding between internet users and makes difficult to use computational approaches since most Arabic resources are intended for the formal language: Modern Standard Arabic (MSA). In this paper, we present a pipeline to standardize the written texts in social networks by translating them to the MSA. We fine-tune at first an identification bert-based model to select Tunisian Dialect (TD) comments from MSA and other dialects. Then, the resulting comments are translated using a neural translation model. Each of these steps was evaluated on the same test corpus. In order to test the effectiveness of the approach, we compared two opinion analysis models, the first is intended for the Sentiment Analysis (SA) of dialect texts and the second is for the MSA texts. We concluded that through standardization we obtain the best score. Keywords: Dialect Identification, Neural Machine Translation, Sentiment Analysis, Tunisian Dialect, Modern Standard Arabic. 1. Introduction The Arab World is a collection of 21 countries cov- ering Arabia, the Near East and North Africa, having in common an Arabic culture and the Arabic language. The Arabic language is marked by an important diglos- sia between the literal Arabic, a mainly written vehic- ular language, and Arabic Dialect (AD), a mainly oral vernacular language. Literal Arabic includes Classi- cal Arabic (pre-Quranic, Quranic, and post-Quranic) and Modern Standard Arabic (MSA). During the last decade, the linguistic situation in the Arab world has been greatly changed. Indeed, with the increasing use of the internet, social media has been invaded by con- tent written with the spontaneous spoken language of each user which is the dialect. This change has led to the appearance of written forms of the AD on internet platforms and has produced gigantic volumes of data written in various dialects. Each of these AD has its linguistic properties due to the absence of a standard norm for its writings. So, the treatment of AD is a bit difficult. In effect, MSA presents the formal lan- guage for all Arabic dialects, but they are different at all linguistic levels. The existing resources of MSA are rendered unable to process the AD. The main issue in this context, is how to understand and treat writing published in social networks to be able to use them in other tasks such as opinion analysis and multiple Nat- ural Language Processing (NLP) related applications. Indeed, two solutions can be envisaged. Either consider a …

Content truncated to 500 words