ESC: Redesigning WSD with Extractive Sense Comprehension

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4661–4672 June 6–11, 2021. ©2021 Association for Computational Linguistics 4661 ESC: Redesigning WSD with Extractive Sense Comprehension Edoardo Barba1 Tommaso Pasini2,∗ Roberto Navigli1 1Sapienza NLP Group, Department of Computer Science, Sapienza University of Rome 2Department of Computer Science, University of Copenhagen {barba,navigli}@di.uniroma1.it tommaso.pasini@di.ku.dk Abstract Word Sense Disambiguation (WSD) is a his- torical NLP task aimed at linking words in contexts to discrete sense inventories and it is usually cast as a multi-label classification task. Recently, several neural approaches have employed sense definitions to better rep- resent word meanings. Yet, these approaches do not observe the input sentence and the sense definition candidates all at once, thus potentially reducing the model performance and generalization power. We cope with this issue by reframing WSD as a span extrac- tion problem — which we called Extractive Sense Comprehension (ESC) — and propose ESCHER, a transformer-based neural architec- ture for this new formulation. By means of an extensive array of experiments, we show that ESC unleashes the full potential of our model, leading it to outdo all of its competi- tors and to set a new state of the art on the English WSD task. In the few-shot scenario, ESCHER proves to exploit training data ef- ficiently, attaining the same performance as its closest competitor while relying on almost three times fewer annotations. Furthermore, ESCHER can nimbly combine data annotated with senses from different lexical resources, achieving performances that were previously out of everyone’s reach. The model along with data is available at https://github.com/ SapienzaNLP/esc. 1 Introduction Being able to link a piece of raw text to a knowl- edge base is fundamental in NLP (Navigli, 2009; McCoy et al., 2019; Bender and Koller, 2020), as it can aid neural models to ground their represen- tations on structured resources and enable Natural Language Understanding (Navigli, 2018). A task that is key to achieving this goal is Word Sense Disambiguation (WSD), where, given a sentence ∗Work carried out while at the Sapienza University of Rome. with a target word, a model has to predict its most suitable meaning from a predefined set of labels, i.e., its senses. WSD has not only considerably improved its performance with the advent of deep learning (by around 15 F1 points in 15 years), but it has also shown its benefits in downstream appli- cations such as Neural Machine Translation (Liu et al., 2018; Pu et al., 2018) and Information Ex- traction (Moro and Navigli, 2013; Delli Bovi et al., 2015), while also being leveraged to enrich the contextual representations of neural models (Peters et al., 2019; Zhang et al., 2019). However, WSD has mostly been framed as a multi-label classifi- cation task (Raganato et al., 2017b; Hadiwinoto et al., 2019) over a very large vocabulary of dis- crete senses. This formulation may limit a model’s capabilities to properly represent word meanings, as each sense is only defined by means …

Content truncated to 500 words