Rare and Zero-shot Word Sense Disambiguation using Z-Reweighting

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4713 - 4723 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Rare and Zero-shot Word Sense Disambiguation using Z-Reweighting Ying Su1, Hongming Zhang1,2, Yangqiu Song1, Tong Zhang1 1HKUST 2Tencent AI lab, Seattle ysuay@connect.ust.hk, {hzhangal,yqsong}@cse.ust.hk, tongzhang@ust.hk Abstract Word sense disambiguation (WSD) is a cru- cial problem in the natural language processing (NLP) community. Current methods achieve decent performance by utilizing supervised learning and large pre-trained language mod- els. However, the imbalanced training dataset leads to poor performance on rare senses and zero-shot senses. There are more training in- stances and senses for words with top frequency ranks than those with low frequency ranks in the training dataset. We investigate the sta- tistical relation between word frequency rank and word sense number distribution. Based on the relation, we propose a Z-reweighting method on the word level to adjust the training on the imbalanced dataset. The experiments show that the Z-reweighting strategy achieves performance gain on the standard English all words WSD benchmark. Moreover, the strat- egy can help models generalize better on rare and zero-shot senses. 1 Introduction Word sense disambiguation (WSD) has been a long- standing problem in natural language processing community. The task can benefit many downstream applications (Navigli, 2009), including but not lim- ited to machine translation (Vickrey et al., 2005; Pu et al., 2018) and information retrieval (Stokoe et al., 2003; Zhong and Ng, 2012). The goal of the WSD task is to disambiguate word senses given contexts. For example, the word “lift” in the context “Lift a load” and “The detective carefully lifted some fingerprints from the table” has different meanings. The former one means “raise from a lower to a higher position” and the latter one means “remove from a surface”. From semantic recognition of human being, the former sense is easier to disambiguate as it is the most common sense of the word while the latter one is a relatively rare one. A skewed distribution exists in SemCor (Miller et al., 1993), a commonly used human-labeled dataset for the WSD task, where most common senses have many training examples while rare senses have much fewer examples. A large cover- age of senses are not accompanied with training examples, which are called zero-shot senses. Many deep neural-networks-based methods are affected by this imbalanced training corpora (Luo et al., 2018; Huang et al., 2019b). Previous approaches attempt to address this prob- lem by designing a new dataset or task specifically for the rare senses and zero-shot senses (Holla et al., 2020; Blevins et al., 2021; Barba et al., 2021) or en- riching the sense embeddings by incorporating ex- ternal lexical knowledge (Kumar et al., 2019; Scar- lini et al., 2020; Blevins and Zettlemoyer, 2020). Different from these methods, we address the un- balanced training issue from the perspective of ad- justing the learning process. An interesting human language phenomenon is that it follows a statistical distribution described by Zipf’s law …

Content truncated to 500 words