Don’t Invite BERT to Drink a Bottle: Modeling the Interpretation of Metonymies Using BERT and Distributional Representations

Proceedings of the 28th International Conference on Computational Linguistics, pages 6831–6837 Barcelona, Spain (Online), December 8-13, 2020 6831 Don’t Invite BERT to Drink a Bottle: Modeling the Interpretation of Metonymies Using BERT and Distributional Representations Paolo Pedinotti Alessandro Lenci CoLing Lab, University of Pisa, Italy pedinotti.paolo@gmail.com alessandro.lenci@unipi.it Abstract In this work, we carry out two experiments in order to assess the ability of BERT to capture the meaning shift associated with metonymic expressions. We test the model on a new dataset that is representative of the most common types of metonymy. We compare BERT with the Structured Distributional Model (SDM), a model for the representation of words in context which is based on the notion of Generalized Event Knowledge. The results reveal that, while BERT ability to deal with metonymy is quite limited, SDM is good at predicting the meaning of metonymic expressions, providing support for an account of metonymy based on event knowledge. 1 Introduction Metonymy is one of the most important sources of lexical polysemy and consists in the meaning shift of a noun that is used to refer to another entity to which it is related (Littlemore, 2015). For instance, bottle refers to a solid container in (1a), but in (1b) it stands for some liquid contained in it: (1) a. The guest broke the bottle. b. The guest tasted the bottle. Metonymy is a productive and systematic process (e.g., all nouns denoting containers show the same pol- ysemy as bottle, giving rise to the so-called CONTAINER-FOR-CONTENT metonymic alternation). There- fore, both linguistic (Pustejovsky, 1995; Jackendoff, 1997; Asher, 2011) and psycholinguistic (Piñango et al., 2016) studies contest the treatment of metonymy like a case of lexical ambiguity, and instead support the hypothesis that metonymic interpretations result from the inherently dynamic and generative nature of lexical representations that can acquire new meanings by integrating information activated by the textual and extralinguistic context. Vector representations (aka word embeddings) produced by Distributional Semantic Models (DSMs) are particularly suitable for modeling contextual semantic effects, due to their “gradedness" and their de- pendence on the linguistic contexts (Lenci, 2018; Boleda, 2020). Traditional DSMs represent the content of lexical types through a single vector that “summarizes” their whole distributional history. Things have recently changed with the introduction of deep neural architectures for language modeling like BERT (Devlin et al., 2019), whose word representations have helped achieving state-of-the-art results in a wide variety of supervised NLP tasks. These embeddings are intrinsically contextualized, in the sense that the model computes a different vector for each token occurrence of the same word, depending on the sentence in which the token appears. In this work, we test whether BERT contextualized embeddings can be used to model the meaning shifts associated with metonymic uses of words. Given its pervasive- ness in everyday communication, we suggest that the extent to which metonymy is captured by BERT is an important testbed to evaluate its actual ability to model natural language. In line with this goal, we require the model to …

Content truncated to 500 words