Latent Vector Weighting for Word Meaning in Context

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1012–1022, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Latent Vector Weighting for Word Meaning in Context Tim Van de Cruys RCEAL University of Cambridge tv234@cam.ac.uk Thierry Poibeau LaTTiCe, UMR8094 CNRS & ENS thierry.poibeau@ens.fr Anna Korhonen Computer Laboratory & RCEAL University of Cambridge alk23@cam.ac.uk Abstract This paper presents a novel method for the com- putation of word meaning in context. We make use of a factorization model in which words, to- gether with their window-based context words and their dependency relations, are linked to latent dimensions. The factorization model al- lows us to determine which dimensions are im- portant for a particular context, and adapt the dependency-based feature vector of the word accordingly. The evaluation on a lexical substi- tution task – carried out for both English and French – indicates that our approach is able to reach better results than state-of-the-art meth- ods in lexical substitution, while at the same time providing more accurate meaning repre- sentations. 1 Introduction According to the distributional hypothesis of meaning (Harris, 1954), words that occur in similar contexts tend to be semantically similar. In the spirit of this by now well-known adage, numerous algorithms have sprouted up that try to capture the semantics of words by looking at their distribution in texts, and compar- ing those distributions in a vector space model. Up till now, the majority of computational ap- proaches to semantic similarity represent the mean- ing of a word as the aggregate of the word’s contexts, and hence do not differentiate between the different senses of a word. The meaning of a word, however, is largely dependent on the particular context in which it appears. Take for example the word work in sen- tences (1) and (2). (1) The painter’s recent work is a classic example of art brut. (2) Equal pay for equal work! The meaning of work is quite different in both sen- tences. In sentence (1), work refers to the product of a creative act, viz. a painting. In sentence (2), it refers to labour carried out as a source of income. The NLP community’s standard answer to the ambiguity problem has always been some flavour of word sense disambiguation (WSD), which in its standard form boils down to choosing the best-possible fit from a pre-defined sense inventory. In recent years, it has become clear that this is in fact a very hard task to solve for computers and humans alike (Ide and Wilks, 2006; Erk et al., 2009; Erk, 2010). With these findings in mind, researchers have started looking at different methods to tackle lan- guage’s ambiguity, ranging from coarser-grained sense inventories (Hovy et al., 2006) and graded sense assignment (Erk and McCarthy, 2009), over word sense induction (Sch¨utze, 1998; Pantel and Lin, 2002; Agirre et al., 2006), to the computation of indi- vidual word meaning in context (Erk and Pad´o, 2008; Thater et al., 2010; Dinu and Lapata, 2010). This …

Content truncated to 500 words