Modelling Lexical Ambiguity with Density Matrices

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 276–290 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 276 Modelling Lexical Ambiguity with Density Matrices Francois Meyer University of Amsterdam francoisrmeyer@gmail.com Martha Lewis University of Bristol, ILLC, University of Amsterdam martha.lewis@bristol.ac.uk Abstract Words can have multiple senses. Composi- tional distributional models of meaning have been argued to deal well with finer shades of meaning variation known as polysemy, but are not so well equipped to handle word senses that are etymologically unrelated, or homonymy. Moving from vectors to density matrices allows us to encode a probability dis- tribution over different senses of a word, and can also be accommodated within a compo- sitional distributional model of meaning. In this paper we present three new neural mod- els for learning density matrices from a cor- pus, and test their ability to discriminate be- tween word senses on a range of compositional datasets. When paired with a particular com- position method, our best model outperforms existing vector-based compositional models as well as strong sentence encoders. 1 Introduction An integral part of natural language understanding is the ability to handle lexical ambiguity. Words can have multiple meanings, and the precise mean- ing of a word only becomes clear when we see it in use - the surrounding context disambiguates it. Word sense disambiguation (WSD) is said to be an ‘AI complete’ problem (Navigli, 2009), that is, a problem that is at least as hard as any other problem in AI, and as such has been the subject of extensive research. Standard approaches treat WSD as a classification problem: given a word in context, the task is to classify it into one of a range of possible senses (Lesk, 1986; Sch¨utze, 1998; Navigli, 2009). A more difficult task is to dis- ambiguate every word in a sentence (Chaplot and Salakhutdinov, 2018). A summary of the state of the art is given in (Raganato et al., 2017). More re- cently, neural approaches (Hadiwinoto et al., 2019; Huang et al., 2019) use contextualised embeddings as input to WSD systems, together with knowledge from WordNet. Other neural approaches gener- ate multiple sense vectors per word (Neelakantan et al., 2014; Cheng and Kartsaklis, 2015) or vectors representing a context (Melamud et al., 2016). Disambiguation can be costly. Each word should be disambiguated with respect to the correct senses of the other words in the sentence, meaning that the computational complexity of the task can become problematic (Chaplot and Salakhutdinov, 2018). Within a compositional framework, the idea is for words to disambiguate automatically in the pro- cess of composition (Kintsch, 2001; Mitchell and Lapata, 2008; Baroni et al., 2014; Boleda, 2020). Within purely vector-based models, the amount of ambiguity that a word vector can represent is limited. Baroni et al. (2014) argue that distribu- tional vectors work well for polysemy, but less so for homonymy. Piedeleu et al. (2015) extend the vector based model of meaning to encompass homonymy by using the notion of a …

Content truncated to 500 words