Word Sense Disambiguation for 158 Languages using Word Embeddings Only

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5943–5952 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5943 Word Sense Disambiguation for 158 Languages using Word Embeddings Only Varvara Logacheva1, Denis Teslenko2, Artem Shelmanov1, Steffen Remus3, Dmitry Ustalov4⋆, Andrey Kutuzov5, Ekaterina Artemova6, Chris Biemann3, Simone Paolo Ponzetto4, Alexander Panchenko1 1Skolkovo Institute of Science and Technology, Moscow, Russia v.logacheva@skoltech.ru 2Ural Federal University, Yekaterinburg, Russia 3Universität Hamburg, Hamburg, Germany 4Universität Mannheim, Mannheim, Germany 5University of Oslo, Oslo, Norway 6Higher School of Economics, Moscow, Russia Abstract Disambiguation of word senses in context is easy for humans, but is a major challenge for automatic approaches. Sophisticated supervised and knowledge-based models were developed to solve this task. However, (i) the inherent Zipfian distribution of supervised training instances for a given word and/or (ii) the quality of linguistic knowledge representations motivate the development of completely unsupervised and knowledge-free approaches to word sense disambiguation (WSD). They are particularly useful for under-resourced languages which do not have any resources for building either supervised and/or knowledge-based models. In this paper, we present a method that takes as input a standard pre-trained word embedding model and induces a fully-fledged word sense inventory, which can be used for disambiguation in context. We use this method to induce a collection of sense inventories for 158 languages on the basis of the original pre-trained fastText word embeddings by Grave et al. (2018), enabling WSD in these languages. Models and system are available online. Keywords: word sense induction, word sense disambiguation, word embeddings, sense embeddings, graph clustering 1. Introduction There are many polysemous words in virtually any lan- guage. If not treated as such, they can hamper the perfor- mance of all semantic NLP tasks (Resnik, 2006). There- fore, the task of resolving the polysemy and choosing the most appropriate meaning of a word in context has been an important NLP task for a long time. It is usually referred to as Word Sense Disambiguation (WSD) and aims at assign- ing meaning to a word in context. The majority of approaches to WSD are based on the use of knowledge bases, taxonomies, and other external man- ually built resources (Moro et al., 2014; Upadhyay et al., 2018). However, different senses of a polysemous word oc- cur in very diverse contexts and can potentially be discrim- inated with their help. The fact that semantically related words occur in similar contexts, and diverse words do not share common contexts, is known as distributional hypoth- esis and underlies the technique of constructing word em- beddings from unlabelled texts. The same intuition can be used to discriminate between different senses of individual words. There exist methods of training word embeddings that can detect polysemous words and assign them differ- ent vectors depending on their contexts (Athiwaratkun et al., 2018; Jain et al., 2019). Unfortunately, many wide- spread word embedding models, such as GloVe (Penning- ton et al., 2014), word2vec (Mikolov et al., 2013), fastText (Bojanowski et al., 2017), …

Content truncated to 500 words