Toward Qualitative Evaluation of Embeddings for Arabic Sentiment Analysis

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4955–4963 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4955 Toward Qualitative Evaluation of Embeddings for Arabic Sentiment Analysis Amira Barhoumi 1,2, Nathalie Camelin1, Chafik Aloulou2, Yannick Est`eve3, Lamia Belguith2 Le Mans University 1, Sfax University2, Avignon University 3 Avenue Laennec, 72085 LE MANS CEDEX 9, France 1, Route de Tunis Km 10 B.P. 242 SFAX 3021, Tunisia 2, Rue Louis Pasteur, 84000 Avignon, France 3 {amira.barhoumi.etu, nathalie.camelin}@univ-lemans.fr {chafik.aloulou, l.belguith}@fsegs.rnu.tn yannick.esteve@univ-avignon.fr Abstract In this paper, we propose several protocols to evaluate specific embeddings for Arabic sentiment analysis (SA) task. In fact, Arabic language is characterized by its agglutination and morphological richness contributing to great sparsity that could affect embedding quality. This work presents a study that compares embeddings based on words and lemmas in SA frame. We propose first to study the evolution of embedding models trained with different types of corpora (polar and non polar) and explore the variation between embeddings by observing the sentiment stability of neighbors in embedding spaces. Then, we evaluate embeddings with a neural architecture based on convolutional neural network (CNN). We make available our pre-trained embeddings to Arabic NLP research community with free to use. We provide also for free resources used to evaluate our embeddings. Experiments are done on the Large Arabic-Book Reviews (LABR) corpus in binary (positive/negative) classification frame. Our best result reaches 91.9%, that is higher than the best previous published one (91.5%). Keywords: Embeddings, Word2vec, Sentiment Analysis, Intrinsic and Extrinsic Evaluation, Convolutional Neural Networks, Arabic language. 1. Introduction Sentiment analysis (SA) task focuses on analysing opin- ions and sentiments at different levels (word, sentence and document) (Nasukawa and Yi, 2003). In this work, we are dealing with SA at document level. It refers to iden- tifying the polarity of a given textual statement (Pang et al., 2008). Generally, polarity consists in associating posi- tive/negative classes to statements (sometimes extended to more fine-grained categories). SA applications have spread many languages, with a recent growing interest in Arabic language. The majority of recent works on Arabic SA lies on word embeddings in order to capture semantic and syntactic simi- larities. Their quality needs large corpora so that each word in the vocabulary appears multiple times in different con- texts. These embeddings deal mainly with words as space separator units. However, Arabic is characterized by its agglutination and morphological richness that contribute to sparsity. So, it is important to take into account Arabic word complexity. The majority of works dealing with continuous representa- tions in Arabic NLP has focused on the word unit. In this work, we target the complex structure of Arabic words in SA task by investigating two ways to build embedding sets: one set by lexical unit (word and lemma). Word embeddings are widely used for various NLP tasks. Embedding evaluation techniques fall into two major cat- egories: intrinsic and extrinsic methods. The aim of this work is to evaluate our proposed …

Content truncated to 500 words