Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3625–3640 August 1–6, 2021. ©2021 Association for Computational Linguistics 3625 Exploring the Representation of Word Meanings in Context: A Case Study on Homonymy and Synonymy Marcos Garcia CiTIUS – Research Center in Intelligent Technologies Universidade de Santiago de Compostela, Galiza marcos.garcia.gonzalez@usc.gal Abstract This paper presents a multilingual study of word meaning representations in context. We assess the ability of both static and contextual- ized models to adequately represent different lexical-semantic relations, such as homonymy and synonymy. To do so, we created a new multilingual dataset that allows us to perform a controlled evaluation of several factors such as the impact of the surrounding context or the overlap between words, conveying the same or different senses. A systematic assessment on four scenarios shows that the best monolingual models based on Transformers can adequately disambiguate homonyms in context. However, as they rely heavily on context, these models fail at representing words with different senses when occurring in similar sentences. Experi- ments are performed in Galician, Portuguese, English, and Spanish, and both the dataset (with more than 3,000 evaluation items) and new models are freely released with this study. 1 Introduction Contrary to static vector models, which represent the different senses of a word in a single vector (Erk, 2012; Mikolov et al., 2013), contextualized models generate representations at token-level (Pe- ters et al., 2018; Devlin et al., 2019), thus being an interesting approach to model word meaning in context. In this regard, several studies have shown that clusters produced by some contextualized word embeddings (CWEs) are related to different senses of the same word (Reif et al., 2019; Wiedemann et al., 2019), or that similar senses can be aligned in cross-lingual experiments (Schuster et al., 2019). However, more systematic evaluations of pol- ysemy (i.e., word forms that have different re- lated meanings depending on the context (Apres- jan, 1974)), have shown that even though CWEs present some correlations with human judgments (Nair et al., 2020), they fail to predict the similarity of the various senses of a polysemous word (Haber and Poesio, 2020). As classical datasets to evaluate the capabilities of vector representations consist of single words without context (Finkelstein et al., 2001) or heavily constrained expressions (Kintsch, 2001; Mitchell and Lapata, 2008), new resources with annotations of words in free contexts have been created, includ- ing both graded similarities (Huang et al., 2012; Armendariz et al., 2020) or binary classification of word senses (Pilehvar and Camacho-Collados, 2019; Raganato et al., 2020). However, as these datasets largely include instances of polysemy, they are difficult to solve even for humans (in fact, the highest reported human upper bound is about 80%) as the nuances between different senses depend on non-linguistic factors such as the annotator pro- cedure or the target task (Tuggy, 1993; Kilgarriff, 1997; Hanks, 2000; Erk, 2010). In this paper, we rely on a more objective and simple …

Content truncated to 500 words