Enhancing the Context Representation in Similarity-based Word Sense Disambiguation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8965–8973 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8965 Abstract In previous similarity-based WSD systems, studies have allocated much effort on learning comprehensive sense embeddings using contextual representations and knowledge sources. However, the context embedding of an ambiguous word is learned using only the sentence where the word appears, neglecting its global context. In this paper, we investigate the contribution of both word-level and sense- level global context of an ambiguous word for disambiguation. Experiments have shown that the Context-Oriented Embedding (COE) can enhance similarity-based system’s performance on a WSD by relatively large margins, achieving state-of-the-art on all-words WSD benchmarks in knowledge-based category. 1 Introduction Word sense disambiguation (WSD) is aimed at selecting the correct sense for a word given its context. Potential senses of a word are from a sense inventory such as WordNet (Miller, 1995). WSD can be classified into lexical sample WSD and all-words WSD. The former focuses on disambiguating some particular words in many sentences, while the latter conducts WSD on every ambiguous word in the provided text. The nature of all-words WSD allows the task to be more compatible to downstream applications. Nevertheless, the task becomes more difficult (Pradhan et al., 2007) while it also provides more context information (rather than a single sentence). * corresponding author Utilizing such global context can assist the systems to tackle WSD from an overall perspective. Recent development of contextual representation models, has accelerated the progress of WSD. Many systems are proposed to tackle WSD by employing BERT either by extracting features (Vial et al., 2019; Loureiro and Jorge, 2019) or fine-tuning (Peters et al., 2019; Levine et al., 2020). However, these systems are mostly implemented with a single sentence context, especially for the systems (Huang et al., 2019; Blevins and Zettlemoyer, 2020) that fine- tune BERT (Devlin et al., 2019). As for the others (Scarlini et al., 2020a; Wang and Wang, 2020, Scarlini et al., 2020b), efforts are allocated to construct sense embeddings using WordNet or SemCor (Miller et al., 1994), while context embeddings for ambiguous words are learned merely from a single sentence. This has led to an issue that the information volume of context embeddings and sense embeddings is not balanced. In this paper, we introduce COE, a context- oriented embedding technique to learn comprehensive context representations for ambiguous words. This is aimed at enhancing the context embeddings by considering both the global and local sentences in the provided document. In summary, our approach has the following contributions: • We propose a novel technique to capture both local and global context information for context representation learning. The obtained Enhancing the Context Representation in Similarity-based Word Sense Disambiguation Ming Wang1, , Jianzhang Zhang2,  and Yinglin Wang3, , * School of Information Management and Engineering, Shanghai University of Finance and Economics Alibaba Business School, Hangzhou Normal University 1wangming@163.sufe.edu.cn, 2jianzhang.zhang@foxmail.com, 3wang.yinglin@shufe.edu.cn 8966 context embeddings are further enhanced with the embeddings of senses appeared in …

Content truncated to 500 words