Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7741–7751 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 7741 Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories Wenlin Yao Xiaoman Pan Lifeng Jin Jianshu Chen Dian Yu Dong Yu Tencent AI Lab, Bellevue, WA, USA {wenlinyao,xiaomanpan,lifengjin,jianshuchen,yudian,dyu}@tencent.com Abstract Word Sense Disambiguation (WSD) aims to automatically identify the exact meaning of one word according to its context. Existing su- pervised models struggle to make correct pre- dictions on rare word senses due to limited training data and can only select the best defini- tion sentence from one predefined word sense inventory (e.g., WordNet). To address the data sparsity problem and generalize the model to be independent of one predefined inventory, we propose a gloss alignment algorithm that can align definition sentences (glosses) with the same meaning from different sense inven- tories to collect rich lexical knowledge. We then train a model to identify semantic equiv- alence between a target word in context and one of its glosses using these aligned invento- ries, which exhibits strong transfer capability to many WSD tasks1. Experiments on bench- mark datasets show that the proposed method improves predictions on both frequent and rare word senses, outperforming prior work by 1.2% on the All-Words WSD Task and 4.3% on the Low-Shot WSD Task. Evaluation on WiC Task also indicates that our method can better capture word meanings in context. 1 Introduction Human language is inherently ambiguous since words can have various meanings in different con- texts. Word Sense Disambiguation (WSD) aims to automatically identify the correct sense (mean- ing) of the target word within a context sentence, which is essential to many downstream tasks such as machine translation and information extraction. Recently, many approaches have achieved state-of- the-art performance on WSD by fine-tuning lan- guage models pretrained with massive text data 1Models and code are available at https://github. com/wenlinyao/EMNLP21-ConnectTheDots. We will also release the checkpoint of the pretrained model for reproducibility. on task-specific datasets (Blevins and Zettlemoyer, 2020; Yap et al., 2020). However, fine-tuning a WSD model using task- specific resources could limit its applicability and may cause two major problems. First, the perfor- mance of models decreases significantly when pre- dicting on rare and zero-shot word senses (Kumar et al., 2019; Choubey and Huang, 2020; Blevins et al., 2021) because there are no sufficient support- ing examples in training data. Second, the trained models are often inventory-dependent which can only select the best definition from one predefined word sense inventory (mainly WordNet) that hu- man annotations are based upon. In this paper, we overcome these limitations by leveraging abundant lexical knowledge from var- ious word sense inventories. As we know, dictio- naries that are compiled by experts contain rich sense knowledge of words. Moreover, a dictio- nary usually provides several example sentences for each word sense to illustrate its usage, which can be viewed as context sentences of that word sense. Since a word’s sense (meaning) can be de- …

Content truncated to 500 words