Combining Lexical Substitutes in Neural Word Sense Induction

Proceedings of Recent Advances in Natural Language Processing, pages 62–70, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_008 62 Combining Lexical Substitutes in Neural Word Sense Induction Nikolay Arefyev1,2, Boris Sheludko1,2, and Alexander Panchenko3 1Samsung R&D Institute Russia, Moscow, Russia 2Lomonosov Moscow State University, Moscow, Russia 3Skolkovo Institute of Science and Technology, Moscow, Russia narefjev@cs.msu.ru, b.sheludko@samsung.com, a.panchenko@skoltech.ru Abstract Word Sense Induction (WSI) is the task of grouping of occurrences of an ambigu- ous word according to their meaning. In this work, we improve the approach to WSI proposed by Amrami and Goldberg (2018) based on clustering of lexical sub- stitutes for an ambiguous word in a par- ticular context obtained from neural lan- guage models. Namely, we propose meth- ods for combining information from left and right context and similarity to the am- biguous word, which result in generating more accurate substitutes than the origi- nal approach. Our simple yet efficient im- provement establishes a new state-of-the- art on WSI datasets for two languages. Besides, we show improvements to the original approach on a lexical substitution dataset. 1 Introduction Ambiguity, including lexical ambiguity, is one of the fundamental properties of natural languages and is a central challenge for NLP and its appli- cations. Lexical ambiguity is a common situation when a single word has several meanings which can be either closely related (coffee as a plant, as a drink, or as beans for preparing that drink) or entirely unrelated (band as a musical group or as a strip of material). Consider the word book in book a flight or buy a book. Depending on the expressed meaning, machine translation systems should translate this word differently, search en- gines should find different information, personal digital assistants should take different actions, etc. Word sense induction (WSI) is the task of clustering of occurrences of an ambiguous word according to their meaning. For evaluation of WSI systems, text fragments containing ambigu- ous words are hand-labeled with senses from some sense inventory (a dictionary or a lexical ontol- ogy). WSI systems are given text fragments only and should cluster them into some a priori un- known number of clusters (unlike Word Sense Disambiguation systems, which are also given the sense inventory). Words that can appear instead of an ambigu- ous word in a particular context, also known as lexical contextual substitutes, are very helpful for WSI because possible substitutes strongly de- pend on the expressed meaning of the ambigu- ous word. For instance, for the word build pos- sible substitutes are manufacture, make, assem- ble, ship, export if it is used in the manufactur- ing some goods sense and erect, rebuild, open for the constructing a building sense. Baskaya et al. (2013) proposed generating substitutes using n-gram language models and had shown one of the best results at the SemEval-2013 WSI shared task for English (Jurgens and Klapaftis, 2013). Later Amrami and Goldberg (2018) proposed generating contextual substitutes with a bidirec- tional neural language model (biLM) ELMo (Pe- ters et al., 2018). With several other improve- …

Content truncated to 500 words