UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4113–4120 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4113 UniSent: Universal Adaptable Sentiment Lexica for 1000+ Languages Ehsaneddin Asgari, Fabienne Braune, Benjamin Roth, Christoph Ringlstetter, Mohammad R.K. Mofrad Department of Bioengineering, University of California, Berkeley, CA 94720, USA NLP Expert Center, Volkswagen Group Data:Lab, Munich, Germany Center for Information and Language Processing, Munich 80538, Germany Molecular Biophysics and Integrated Bioimaging, Lawrence Berkeley National Lab, Berkeley, CA 94720, USA inquiries@language-lab.info Abstract In this paper, we introduce UniSent universal sentiment lexica for 1000+ languages. Sentiment lexica are vital for sentiment analysis in absence of document-level annotations, a very common scenario for low-resource languages. To the best of our knowledge, UniSent is the largest sentiment resource to date in terms of the number of covered languages, including many low resource ones. In this work, we use a massively parallel Bible corpus to project sentiment information from English to other languages for sentiment analysis on Twitter data. We introduce a method called DomDrift to mitigate the huge domain mismatch between Bible and Twitter by a confidence weighting scheme that uses domain-specific embeddings to compare the nearest neighbors for a candidate sentiment word in the source (Bible) and target (Twitter) domain. We evaluate the quality of UniSent in a subset of languages for which manually created ground truth was available, Macedonian, Czech, German, Spanish, and French. We show that the quality of UniSent is comparable to manually created sentiment resources when it is used as the sentiment seed for the task of word sentiment prediction on top of embedding representations. In addition, we show that emoticon sentiments could be reliably predicted in the Twitter domain using only UniSent and monolingual embeddings in German, Spanish, French, and Italian. With the publication of this paper, we release the UniSent sentiment lexica at http://language-lab.info/unisent. Keywords: Universal Sentiment Lexicon, 1000+ Languages, Embedding-based Domain Shift Detection 1. Introduction Language technologies permeate our everyday life through web search, translation, online shopping, email writing, spell checking systems etc. The existence of such technolo- gies highly depends on the existence of the underlying com- putational linguistic resources for a language. Computa- tional linguistic resources such as machine-readable lexica, part-of-speech-taggers and dependency parsers are avail- able for at most a few hundred languages. This means that the majority of the 7000 languages of the world are low- resource. This gap between advances in language technolo- gies for English versus other languages endangers multilin- gualism in the digital age. Languages with lack of tech- nological support (as a result of having limited resources) are less used over time and eventually get in danger of ex- tinction. Many EU and US programs are designed to ad- dress this issue (Cieri et al., 2016). The rationale of these projects is that even “small” languages are important for the preservation of the common heritage of humankind and cultural diversity which benefits everybody. In addition, certain low-resource languages can be …

Content truncated to 500 words