ConSeC: Word Sense Disambiguation as Continuous Sense Comprehension

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1492–1503 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1492 ConSeC: Word Sense Disambiguation as Continuous Sense Comprehension Edoardo Barba and Luigi Procopio and Roberto Navigli Sapienza NLP Group, Sapienza University of Rome {edoardo.barba, luigi.procopio, roberto.navigli}@uniroma1.it Abstract Supervised systems have nowadays become the standard recipe for Word Sense Disam- biguation (WSD), with Transformer-based language models as their primary ingredient. However, while these systems have certainly attained unprecedented performances, virtu- ally all of them operate under the constraining assumption that, given a context, each word can be disambiguated individually with no ac- count of the other sense choices. To address this limitation and drop this assumption, we propose CONtinuous SEnse Comprehension (CONSEC), a novel approach to WSD: lever- aging a recent re-framing of this task as a text extraction problem, we adapt it to our for- mulation and introduce a feedback loop strat- egy that allows the disambiguation of a target word to be conditioned not only on its con- text but also on the explicit senses assigned to nearby words. We evaluate CONSEC and examine how its components lead it to sur- pass all its competitors and set a new state of the art on English WSD. We also explore how CONSEC fares in the cross-lingual set- ting, focusing on 8 languages with various de- grees of resource availability, and report signif- icant improvements over prior systems. We re- lease our code at https://github.com/ SapienzaNLP/consec. 1 Introduction Being able to understand the meaning of the vari- ous words within a particular text is a crucial prob- lem in Natural Language Processing (NLP), with Word Sense Disambiguation (WSD) as arguably its most famous framing: given a word in context, this task aims to pair it with its most suitable meaning, chosen from a fixed sense inventory (Bevilacqua et al., 2021). Similarly to the vast majority of other NLP tasks, the advent of Deep Learning has signif- icantly affected the landscape of WSD and has led to supervised neural models becoming its primary actors. In their simplest flavour, these approaches essentially frame this task as a multi-label classifi- cation problem over a large vocabulary of discrete senses (Raganato et al., 2017b; Hadiwinoto et al., 2019). However, although effective and straight- forward, this formulation suffers from a number of pitfalls, most notably i) senses are only defined via their training set occurrences, with their actual linguistic meaning not explicitly embedded within the neural model, and ii) these architectures either behave poorly on rare and unseen senses, or can- not handle them at all. In order to address these issues, recent literature has proposed more sophis- ticated forms of supervision where definitions of senses, i.e. glosses (Kumar et al., 2019; Blevins and Zettlemoyer, 2020), and relational knowledge coming from the sense inventory (Bevilacqua and Navigli, 2020; Conia and Navigli, 2021) are inte- grated within the neural models. Although performances have been rising steadily and are now beyond the 80% barrier on …

Content truncated to 500 words