A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3506–3517 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3506 A Multilingual BPE Embedding Space for Universal Sentiment Lexicon Induction Mengjie Zhao and Hinrich Sch¨utze CIS, LMU Munich, Germany mzhao@cis.lmu.de Abstract We present a new method for sentiment lex- icon induction that is designed to be appli- cable to the entire range of typological di- versity of the world’s languages. We eval- uate our method on Parallel Bible Corpus+ (PBC+), a parallel corpus of 1593 languages. The key idea is to use Byte Pair Encodings (BPEs) as basic units for multilingual em- beddings. Through zero-shot transfer from English sentiment, we learn a seed lexicon for each language in the domain of PBC+. Through domain adaptation, we then gener- alize the domain-specific lexicon to a general one. We show – across typologically diverse languages in PBC+ – good quality of seed and general-domain sentiment lexicons by intrin- sic and extrinsic and by automatic and human evaluation. We make freely available our code, seed sentiment lexicons for all 1593 languages and induced general-domain sentiment lexi- cons for 200 languages.1 1 Introduction Lexicons play an important role in sentiment anal- ysis. Sentiment lexicons are available for high- resource languages like English (Pang et al., 2008; Baccianella et al., 2010; Mohammad and Tur- ney, 2013), but not for many low-resource lan- guages. Researchers are trying to fill this gap by inducing lexicons monolingually (Badaro et al., 2014; Eskander and Rambow, 2015; Rouces et al., 2018) as well as multilingually (Chen and Skiena, 2014), often by transfer from high-resource to low-resource languages. The world’s languages are heterogeneous – of particular relevance for us is heterogeneity with respect to morphology and with respect to mark- ing token boundaries. This heterogeneity poses difficulties when designing a universal approach 1cistern.cis.lmu.de to lexicon induction that works for all languages – implementing a high quality tokenizer and mor- phological analyzer for each language is not fea- sible short-term. Given the small number of native speakers in low-resource languages (Gold- hahn et al., 2016), crowdsourcing cannot easily be carried out either. To overcome this heterogeneity and provide sentiment resources for low-resource languages, we present a new approach to sentiment lexicon induction that is universal – that is, it is appli- cable to the full range of typologically different languages – and apply it to 1593 languages. Our method first takes a parallel corpus as input and applies BPE (Gage, 1994) segmentation to it. We then create a multilingual BPE embedding space, from which a ZS (zero-shot) lexicon for each lan- guage L is extracted by zero-shot transfer from English sentiment to L. We use PBC+, an ex- pansion of the Parallel Bible Corpus (Mayer and Cysouw, 2014), as our parallel corpus. The ZS lexicons show high quality, but are specific to the domain of PBC+ (the Bible). We then adapt them to the general domain. For brevity, we also use generic to refer …

Content truncated to 500 words