Separating Disambiguation from Composition in Distributional Semantics

Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 114–123, Sofia, Bulgaria, August 8-9 2013. c⃝2013 Association for Computational Linguistics Separating Disambiguation from Composition in Distributional Semantics Dimitri Kartsaklis University of Oxford Dept of Computer Science Wolfson Bldg, Parks Road Oxford, OX1 3QD, UK dimitri.kartsaklis@cs.ox.ac.uk Mehrnoosh Sadrzadeh Queen Mary Univ. of London School of Electr. Engineering and Computer Science Mile End Road London, E1 4NS, UK mehrs@eecs.qmul.ac.uk Stephen Pulman University of Oxford Dept of Computer Science Wolfson Bldg, Parks Road Oxford, OX1 3QD, UK stephen.pulman@cs.ox.ac.uk Abstract Most compositional-distributional models of meaning are based on ambiguous vec- tor representations, where all the senses of a word are fused into the same vec- tor. This paper provides evidence that the addition of a vector disambiguation step prior to the actual composition would be beneficial to the whole process, produc- ing better composite representations. Fur- thermore, we relate this issue with the current evaluation practice, showing that disambiguation-based tasks cannot reli- ably assess the quality of composition. Us- ing a word sense disambiguation scheme based on the generic procedure of Schütze (1998), we first provide a proof of con- cept for the necessity of separating dis- ambiguation from composition. Then we demonstrate the benefits of an “unambigu- ous” system on a composition-only task. 1 Introduction Compositional and distributional semantic mod- els seem to provide complementary solutions for solving the same problem, that of assigning a proper “meaning” to a text segment. Specifically, while compositional models deal with the recur- sive nature of the language, providing a way to address its inherent ability to create infinite sen- tences from finite resources (words), they leave words as unexplained primitives whose meanings have somehow already been set before the compo- sitional process. On the other hand, distributional models have been especially successful in provid- ing concrete representations for the meaning of words as vectors in a vector space, created by tak- ing into account the context in which each word appears. Despite its success for smaller language units, the distributional hypothesis does not natu- rally lend itself to compounds of words. Hence these models do not canonically scale in tasks re- quiring the creation of vector representations for text constituents larger than words, i.e. for phrases and sentences. Given the complementary nature of those two semantic models, it is not surprising that consider- able research activity has been dedicated on com- bining them into a single framework that would benefit from the best of both worlds in a uni- fied manner: Mitchell and Lapata (2008) exper- iment with intransitive sentences, applying sim- ple compositional models based on vector ad- dition and point-wise multiplication in a disam- biguation task; Baroni and Zamparelli (2010) and Guevara (2010) use regression models in order to build vectors for adjective-noun compounds; Erk and Padó (2008) work on transitive sentences us- ing structured vector spaces; Socher et al. (2010, 2011, 2012) use neural networks to combine vec- tors following the grammatical structure; Grefen- stette and Sadrzadeh (2011a,b) apply the categori- …

Content truncated to 500 words