Semantic Specialization for Knowledge-based Word Sense Disambiguation

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3457–3470 May 2-6, 2023 ©2023 Association for Computational Linguistics Semantic Specialization for Knowledge-based Word Sense Disambiguation Sakae Mizuki and Naoaki Okazaki Tokyo Institute of Technology {sakae.mizuki@nlp., okazaki@}c.titech.ac.jp Abstract A promising approach for knowledge-based Word Sense Disambiguation (WSD) is to select the sense whose contextualized embeddings computed for its definition sentence are closest to those computed for a target word in a given sentence. This approach relies on the similarity of the sense and context embeddings computed by a pre-trained language model. We propose a semantic specialization for WSD where contex- tualized embeddings are adapted to the WSD task using solely lexical knowledge. The key idea is, for a given sense, to bring semanti- cally related senses and contexts closer and send different/unrelated senses farther away. We realize this idea as the joint optimization of the Attract-Repel objective for sense pairs and the self-training objective for context-sense pairs while controlling deviations from the orig- inal embeddings. The proposed method outper- formed previous studies that adapt contextual- ized embeddings. It achieved state-of-the-art performance on knowledge-based WSD when combined with the reranking heuristic that uses the sense inventory. We found that the similar- ity characteristics of specialized embeddings conform to the key idea. We also found that the (dis)similarity of embeddings between the re- lated/different/unrelated senses correlates well with the performance of WSD. 1 Introduction Word Sense Disambiguation (WSD) is the task of choosing the appropriate sense of a word from a given sense inventory using contextual informa- tion. WSD has proven its usefulness for Informa- tion Retrieval (Zhong and Ng, 2012) and Machine Translation (Campolungo et al., 2022). A series of extensive studies has led supervised WSD task performance to surpass the milestone of 80% accu- racy (Bevilacqua and Navigli, 2020), which is the estimated human performance (Navigli, 2009). In contrast, the goal of this study is knowledge- based WSD: a variant of WSD that does not rely on supervision data but only on lexical knowledge (e.g., word ontology). This task setting is practi- cally appealing because it does not use a corpus with sense annotations (Bevilacqua et al., 2021), which is costly and labor-intensive to prepare. A promising approach is based on similarity: to select the sense that is the nearest to a target word in the embedding space (Wang and Wang, 2020). Specifically, a pre-trained language model, typically BERT (Devlin et al., 2019), is used to compute sense embeddings for definition sentences. Similarly, a target word is encoded into a context embedding for a given sentence. Then, the model predicts the sense of the target word by finding the most similar sense embedding to the context. The inherent challenge of the similarity-based approach is how we associate two different repre- sentations of word meanings, either by definition sentences or by words in context. Although the BERT embeddings capture the coarse-grained word meanings (Reif et al., 2019; Loureiro et al., 2021), there should be room …

Content truncated to 500 words