Sentiment after Translation: A Case-Study on Arabic Social Media Posts

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 767–777, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Sentiment after Translation: A Case-Study on Arabic Social Media Posts Mohammad Salameh University of Alberta msalameh@ualberta.ca Saif M. Mohammad and Svetlana Kiritchenko National Research Council Canada {saif.mohammad,svetlana.kiritchenko} @nrc-cnrc.gc.ca Abstract When text is translated from one language into another, sentiment is preserved to varying degrees. In this paper, we use Arabic social media posts as stand-in for source language text, and determine loss in sentiment pre- dictability when they are translated into En- glish, manually and automatically. As bench- marks, we use manually and automatically de- termined sentiment labels of the Arabic texts. We show that sentiment analysis of English translations of Arabic texts produces compet- itive results, w.r.t. Arabic sentiment analy- sis. We discover that even though translation significantly reduces the human ability to re- cover sentiment, automatic sentiment systems are still able to capture sentiment information from the translations. 1 Introduction Automatic sentiment analysis of text, especially so- cial media posts, has a number of applications in commerce, public health, and public policy devel- opment. However, a vast majority of prior research on automatic sentiment analysis has been on En- glish texts. Furthermore, many sentiment resources essential to automatic sentiment analysis (e.g., sen- timent lexicons) exist only in English. Thus there is a growing need for effective methods for analyz- ing text from other languages such as Arabic and Chinese, especially posts on social media. There has also been marked progress in automatic trans- lation of texts, especially from other languages into English. Thus, instead of building source-language specific sentiment analysis systems, one can trans- late the texts into English and use an English sen- timent analysis system. However, it is widely be- lieved that aspects of sentiment may be lost in trans- lation, especially in automatic translation. Though, the extent of this loss, in terms of drop in accuracy of automatic sentiment systems remains undetermined. This paper analyzes several methods available in annotating non-English texts for sentiment: • Use a source-language sentiment analysis sys- tem. • Run an English sentiment analysis system on manually created English translations of source language text. • Run an English sentiment analysis system on automatically generated English translations of source language text. In our experiments, we use Arabic social media posts as a specific instance of the source language text. We use state-of-the-art Arabic and English sen- timent analysis systems as well as a state-of-the- art Arabic-to-English translation system. We out- line the advantages and disadvantages of each of the methods listed above, and more importantly con- duct experiments to determine accuracy of sentiment labels obtained using each of these methods. As benchmarks we use manually and automatically de- termined sentiment labels of the Arabic tweets. These results will help users determine methods best suited for their particular needs. Along the way, we answer several research questions such as: 1. What sentiment prediction …

Content truncated to 500 words