Contextualizing Semantic Representations Using Syntactically Enriched Vector Models

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 948–957, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Contextualizing Semantic Representations Using Syntactically Enriched Vector Models Stefan Thater and Hagen Fürstenau and Manfred Pinkal Department of Computational Linguistics Saarland University {stth, hagenf, pinkal}@coli.uni-saarland.de Abstract We present a syntactically enriched vec- tor model that supports the computation of contextualized semantic representations in a quasi compositional fashion. It em- ploys a systematic combination of first- and second-order context vectors. We apply our model to two different tasks and show that (i) it substantially outperforms previ- ous work on a paraphrase ranking task, and (ii) achieves promising results on a word- sense similarity task; to our knowledge, it is the first time that an unsupervised method has been applied to this task. 1 Introduction In the logical paradigm of natural-language seman- tics originating from Montague (1973), semantic structure, composition and entailment have been modelled to an impressive degree of detail and formal consistency. These approaches, however, lack coverage and robustness, and their impact on realistic natural-language applications is lim- ited: The logical framework suffers from over- specificity, and is inappropriate to model the per- vasive vagueness, ambivalence, and uncertainty of natural-language semantics. Also, the hand- crafting of resources covering the huge amounts of content which are required for deep semantic processing is highly inefficient and expensive. Co-occurrence-based semantic vector models of- fer an attractive alternative. In the standard ap- proach, word meaning is represented by feature vectors, with large sets of context words as dimen- sions, and their co-occurrence frequencies as val- ues. Semantic similarity information can be ac- quired using unsupervised methods at virtually no cost, and the information gained is soft and gradual. Many NLP tasks have been modelled successfully using vector-based models. Examples include in- formation retrieval (Manning et al., 2008), word- sense discrimination (Schütze, 1998) and disam- biguation (McCarthy and Carroll, 2003), to name but a few. Standard vector-space models have serious lim- itations, however: While semantic information is typically encoded in phrases and sentences, distri- butional semantics, in sharp contrast to logic-based semantics, does not offer any natural concept of compositionality that would allow the semantics of a complex expression to be computed from the meaning of its parts. A different, but related prob- lem is caused by word-sense ambiguity and con- textual variation of usage. Frequency counts of context words for a given target word provide in- variant representations averaging over all different usages of the target word. There is no obvious way to distinguish the different senses of e.g. acquire in different contexts, such as acquire knowledge or acquire shares. Several approaches for word-sense disambigua- tion in the framework of distributional semantics have been proposed in the literature (Schütze, 1998; McCarthy and Carroll, 2003). In contrast to these approaches, we present a method to model the mu- tual contextualization of words in a phrase in a com- positional way, guided by syntactic structure. To some extent, our method resembles the …

Content truncated to 500 words