Don’t Neglect the Obvious: On the Role of Unambiguous Words in Word Sense Disambiguation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3514–3520, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3514 Don’t Neglect the Obvious: On the Role of Unambiguous Words in Word Sense Disambiguation Daniel Loureiro LIAAD - INESC TEC University of Porto, Portugal dloureiro@fc.up.pt Jose Camacho-Collados School of Computer Science and Informatics Cardiff University, United Kingdom camachocolladosj@cardiff.ac.uk Abstract State-of-the-art methods for Word Sense Dis- ambiguation (WSD) combine two different features: the power of pre-trained language models and a propagation method to extend the coverage of such models. This propaga- tion is needed as current sense-annotated cor- pora lack coverage of many instances in the underlying sense inventory (usually WordNet). At the same time, unambiguous words make for a large portion of all words in WordNet, while being poorly covered in existing sense- annotated corpora. In this paper, we propose a simple method to provide annotations for most unambiguous words in a large corpus. We in- troduce the UWA (Unambiguous Word Anno- tations) dataset and show how a state-of-the- art propagation-based model can use it to ex- tend the coverage and quality of its word sense embeddings by a significant margin, improv- ing on its original results on WSD. 1 Introduction There has been a lot of progress in word sense dis- ambiguation (WSD) recently. This progress has been driven by two factors: (1) the introduction of large pre-trained Transformer-based language mod- els and (2) propagation algorithms that extends the coverage of existing training sets. The gains due to pre-trained Neural Language Models (NLMs) such as BERT (Devlin et al., 2019) have been out- standing, helping reach levels close to human per- formance when training data is available. These models are generally based on a nearest neighbours strategy, where each sense is represented by a vec- tor, exploiting the contextualized embeddings of these NLMs (Melamud et al., 2016; Peters et al., 2018; Loureiro and Jorge, 2019). However, train- ing data for WSD is hard to obtain, and the most widely used training set nowadays, based on Word- Net, dates back from the 90s (Miller et al., 1993, SemCor). This lack of curated data produces the so-called knowledge-acquisition bottleneck (Gale et al., 1992; Navigli, 2009). However, there is a key source of information that has been neglected so far in existing sense- annotated corpora and propagation methods, which is the presence of unambiguous words from the underlying knowledge resource. Strikingly, Word- Net, which is known to be a comprehensive re- source, is mostly composed of unambiguous en- tries (30k lemmas are ambiguous, compared to 116k unambiguous). While the lack of unambigu- ous annotations does not have a direct effect in WSD, the fact that these unambiguous words are part of the same semantic network means they can have an effect on ambiguous words via standard propagation algorithms. These propagation algo- rithms start from a seed of senses occurring in the training data (and therefore their embeddings can be directly computed) and then propagate to the whole sense inventory …

Content truncated to 500 words