Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5218–5229 August 1–6, 2021. ©2021 Association for Computational Linguistics 5218 Abstract Lately proposed Word Sense Disambiguation (WSD) systems have approached the estimated upper bound of the task on standard evaluation benchmarks. However, these systems typically a implement the disambiguation of words in document almost independently, underutilizing sense and word dependency in context. In this paper, we convert the nearly isolated decisions into interrelated ones by exposing senses in context when learning sense embeddings in a similarity- based Sense Aware Context Exploitation (SACE) architecture. Meanwhile, we enhance the context embedding learning with selected sentences from the same document, rather than utilizing only the sentence where each ambiguous word appears. Experiments on both English and multilingual WSD datasets have shown the effectiveness of our approach, surpassing previous state-of-the-art by large margins (3.7% and 1.2% respectively), especially on few-shot (14.3%) and zero-shot (35.9%) scenarios. 1 Introduction Word Sense Disambiguation (WSD) is the task of determining a word’s sense given its context. Recently, contextualized representation learning (Devlin et al., 2019; Liu et al., 2019) have accelerated the advancement of WSD, raising the performance on a standard evaluation framework (Raganato et al., 2017a) from slightly higher than 70% (Raganato et al., 2017b; Luo et al., 2018; Kumar et al., 2019) to about 80% (Vial et al., 2019; Blevins and Zettlemoyer, 2020; Bevilacqua and * corresponding author Navigli, 2020). This is an estimated upper bound of the task, which is from the inter-annotator agreement: the percentage of words that are annotated with the same meaning by two or more annotators (Navigli, 2009). There is a clear trend that supervised systems tend to incorporate sense knowledge into their architecture, ranging from sense definition, usage examples to sense relation. However, the disambiguation of words in a document is almost independent of each other, especially from the perspective of senses in context. The connection of each word’s disambiguation is limited to the utilization of a sentence (Loureiro and Jorge, 2019; Huang et al., 2019; Hadiwinoto et al., 2019; Scarlini et al., 2020a) or a small window of text (Bevilacqua and Navigli, 2020) because of computation cost or model restriction. More severely, the interaction of senses in context is barely explored. Similar to word cooccurrence, the appearance of one sense can sometimes dominate the choice of another sense in the same sentence (Agirre et al., 2014; Maru et al., 2019). In this paper, we introduce SACE, a similarity- based WSD approach. Precisely, we transform the previously almost isolated disambiguation of words in a document into interrelated ones to maximize the contribution of context from both word and sense perspectives. We summarize our contributions as follows: 1. We devise an interactive sense embedding learning technique that takes into account senses in context via a selective attention layer in a neural architecture. It connects senses via their appearance in a piece of text rather than using manually constructed sense …

Content truncated to 500 words