Estimating Linear Models for Compositional Distributional Semantics

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1263–1271, Beijing, August 2010 Estimating Linear Models for Compositional Distributional Semantics Fabio Massimo Zanzotto1 (1) Department of Computer Science University of Rome “Tor Vergata” zanzotto@info.uniroma2.it Ioannis Korkontzelos Department of Computer Science University of York johnkork@cs.york.ac.uk Francesca Fallucchi1,2 (2) Universit`a Telematica “G. Marconi” f.fallucchi@unimarconi.it Suresh Manandhar Department of Computer Science University of York suresh@cs.york.ac.uk Abstract In distributional semantics studies, there is a growing attention in compositionally determining the distributional meaning of word sequences. Yet, compositional dis- tributional models depend on a large set of parameters that have not been explored. In this paper we propose a novel approach to estimate parameters for a class of com- positional distributional models: the addi- tive models. Our approach leverages on two main ideas. Firstly, a novel idea for extracting compositional distributional se- mantics examples. Secondly, an estima- tion method based on regression models for multiple dependent variables. Experi- ments demonstrate that our approach out- performs existing methods for determin- ing a good model for compositional dis- tributional semantics. 1 Introduction Lexical distributional semantics has been largely used to model word meaning in many fields as computational linguistics (McCarthy and Carroll, 2003; Manning et al., 2008), linguistics (Harris, 1964), corpus linguistics (Firth, 1957), and cogni- tive research (Miller and Charles, 1991). The fun- damental hypothesis is the distributional hypoth- esis (DH): “similar words share similar contexts” (Harris, 1964). Recently, this hypothesis has been operationally defined in many ways in the fields of physicology, computational linguistics, and infor- mation retrieval (Li et al., 2000; Pado and Lapata, 2007; Deerwester et al., 1990). Given the successful application to words, dis- tributional semantics has been extended to word sequences. This has happened in two ways: (1) via the reformulation of DH for specific word se- quences (Lin and Pantel, 2001); and (2) via the definition of compositional distributional seman- tics (CDS) models (Mitchell and Lapata, 2008; Jones and Mewhort, 2007). These are two differ- ent ways of addressing the problem. Lin and Pantel (2001) propose the pattern dis- tributional hypothesis that extends the distribu- tional hypothesis for specific patterns, i.e. word sequences representing partial verb phrases. Dis- tributional meaning for these patterns is derived directly by looking to their occurrences in a cor- pus. Due to data sparsity, patterns of different length appear with very different frequencies in the corpus, affecting their statistics detrimentally. On the other hand, compositional distributional semantics (CDS) propose to obtain distributional meaning for sequences by composing the vectors of the words in the sequences (Mitchell and Lap- ata, 2008; Jones and Mewhort, 2007). This ap- proach is fairly interesting as the distributional meaning of sequences of different length is ob- tained by composing distributional vectors of sin- gle words. Yet, many of these approaches have a large number of parameters that cannot be easily estimated. In this paper we propose a novel approach to es- 1263 timate parameters for additive compositional dis- tributional semantics models. Our approach lever- ages on two …

Content truncated to 500 words