Lacking the Embedding of a Word? Look it up into a Traditional Dictionary

Findings of the Association for Computational Linguistics: ACL 2022, pages 2651 - 2662 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Lacking the Embedding of a Word? Look it up into a Traditional Dictionary Elena Sofia Ruzzetti University of Rome Tor Vergata, Italy Leonardo Ranaldi Guglielmo Marconi University, Italy Michele Mastromattei Campus Bio-Medico University, Italy University of Rome Tor Vergata, Italy Francesca Fallucchi Guglielmo Marconi University, Italy Noemi Scarpato San Raffaele Roma Open University, Italy Fabio Massimo Zanzotto University of Rome Tor Vergata, Italy Abstract Word embeddings are powerful dictionaries, which may easily capture language variations. However, these dictionaries fail to give sense to rare words, which are surprisingly often cov- ered by traditional dictionaries. In this paper, we propose to use definitions retrieved in tra- ditional dictionaries to produce word embed- dings for rare words. For this purpose, we intro- duce two methods: Definition Neural Network (DefiNNet) and Define BERT (DefBERT). In our experiments, DefiNNet and DefBERT sig- nificantly outperform state-of-the-art as well as baseline methods devised for producing em- beddings of unknown words. In fact, DefiNNet significantly outperforms FastText, which im- plements a method for the same task-based on n-grams, and DefBERT significantly outper- forms the BERT method for OOV words. Then, definitions in traditional dictionaries are useful to build word embeddings for rare words. 1 Introduction Words without meaning are like compasses without needles: pointless. Indeed, meaningless words lead compositionally to meaningless sentences and, con- sequently, to meaningless texts and conversations. Second language learners may grasp grammatical structures of sentences, but, if they are unaware of the meaning of single words in these sentences, they may fail to understand the whole sentences, especially when there is an insufficient context for unfamiliar words. This is why a large body of natural language processing research is devoted to devising ways to capture word meaning. As language is a living body, distributional meth- ods (Turney and Pantel, 2010; Mikolov et al., 2013; Pennington et al., 2014) are seen as the panacea to capture word meaning as opposed to more static models based on dictionaries (Fellbaum, 1998) and other lexical resources (Baker et al., 1998; Kip- per et al., 2000). Distributional methods may eas- ily capture new meaning of existing words and, eventually, can easily assign meaning to emerging words. In fact, the different methods can scan cor- pora and derive the meaning of these new words by observing them in context (Harris, 1954; Firth, 1950; Wittgenstein, 1953). Words are then repre- sented as vectors – now called word embeddings – which are then used to feed neural networks to pro- duce meaning for sentences (Bengio et al., 2003; ˙Irsoy and Cardie, 2014; Kalchbrenner et al., 2014; Tai et al., 2015) and meaning for whole texts (Joulin et al., 2017; Lai et al., 2015). Distributional methods have a strong limitation: word meaning can be assigned only for words where sufficient contexts can be gathered. Rare words are not covered and become the classical out-of-vocabulary words, which may hinder the understanding of …

Content truncated to 500 words