Integrating Personalized PageRank into Neural Word Sense Disambiguation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9092–9098 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9092 Integrating Personalized PageRank into Neural Word Sense Disambiguation Ahmed ElSheikh Michele Bevilacqua Roberto Navigli Sapienza NLP Group Sapienza University of Rome firstname.lastname@uniroma1.it Abstract Neural Word Sense Disambiguation (WSD) has recently been shown to benefit from the incorporation of pre-existing knowledge, such as that coming from the WordNet graph. How- ever, state-of-the-art approaches have been successful in exploiting only the local struc- ture of the graph, with only close neighbors of a given synset influencing the prediction. In this work, we improve a classification model by recomputing logits as a function of both the vanilla independently produced logits and the global WordNet graph. We achieve this by incorporating an online neural approximated PageRank, which enables us to refine edge weights as well. This method exploits the global graph structure while keeping space requirements linear in the number of edges. We obtain strong improvements, matching the current state of the art. Code is available at https://github.com/SapienzaNLP/ neural-pagerank-wsd. 1 Introduction Word Sense Disambiguation (WSD) is a task with a long history in Natural Language Processing (Bevilacqua et al., 2021). It addresses the perva- sive phenomenon of (lexical) ambiguity, whereby the same polysemous word or expression conveys different meanings in different contexts. Natural Language Processing (NLP) tends to maintain the working assumption that word meaning can be dis- cretized in a finite number of classes, thus casting polysemy resolution as a multi-class classification problem, where the classes, i.e., the senses, are spe- cific to a word. Senses are registered in a dictionary- like resource called the sense inventory. In English WSD the sense inventory is virtually always Word- Net (Miller et al., 1990), which, for example, lists separately the fish and musical instrument senses of the word “bass”. The WSD task is currently dominated by su- pervised methods (Yap et al., 2020; Blevins and Zettlemoyer, 2020). These methods, thanks, in- ter alia, to the game-changing effect of pretrained language models, have widened the margin over so-called knowledge-based approaches (Agirre and Soroa, 2009; Moro et al., 2014; Scozzafava et al., 2020), which usually disambiguate using only global graph information – a source of informa- tion that, however, is not easy to integrate explictly into supervised WSD. Although there have been a few successful approaches integrating graphs into a standard neural classification architecture (Conia and Navigli, 2021; Bevilacqua and Navigli, 2020), such methods only exploit the local relational struc- ture, leaving the global structure unused. Thus, while the model is able to directly utilize the ex- plicit knowledge that a cairn is a terrier, it is not able to also utilize the fact that, following the hy- ponymy relation, a cairn is also a dog. In this paper we propose a method for integrat- ing global graph information into neural supervised WSD through a Personalized PageRank (Page et al., 1999, PPR) approximation, blurring the distinction between knowledge-based and supervised methods. We achieve this …

Content truncated to 500 words