An Empirical Investigation into Grammatically Constrained Contexts in Predicting Distributional Similarity

An Empirical Investigation into Grammatically Constrained Contexts in Predicting Distributional Similarity Dongqiang Yang | David M.W. Powers School of Informatics and Engineering Flinders University of South Australia Po Box 2100, Adelaide 5001, South Australia {Dongqiang.Yang|David.Powers@flinders.edu.au} Abstract The utility of syntactic dependencies in computing distributional similarity has not yet been fully investigated. Most re- search based on syntactically conditioned co-occurrences simply ignores the sali- ence of grammatical relations and effec- tively merges syntactic dependencies into one ‘context’. Through calculating distri- butional similarity, we design two ex- periments to explore and evaluate the four major types of contexts that are con- ditioned on grammatical relations. The consistent results show that the head- modifier dependency plays an important role in predicting the semantic features of nouns and verbs, in contrast to other de- pendencies. 1 Introduction The roles of grammatical relations in predicting semantic similarity via distributional similarity have not been fully analysed. Most approaches simply chained these syntactic dependencies into one unified context representation for computing distributional similarity, such as in Word Sense Disambiguation (WSD) (Yarowsky, 1993; Lin, 1997; Resnik, 1997), word sense induction (Pantel and Lin, 2002), automatic thesaurus con- struction (Grefenstette, 1992; Lin, 1998; Curran, 2003), finding the predominant sense (McCarthy et al., 2004), etc. It is clear that these approaches weighed each dependency through its frequency statistics, e.g. in the weighted (Grefenstette, 1992) or mutual information based (Lin, 1998) Jaccard’s coeffi- cient. Although they proposed to replace the un- ordered context with the syntactically condi- tioned one, the linguistic specificity of gram- matical relations in semantics prediction is often overlooked. Except for the extraction of syntacti- cally conditioned contexts, they in fact make no differentiation between grammatical relations, which work analogously as computing distribu- tional similarity with unordered contexts. With- out distinguishing the linguistic specificity of grammatical relations, the advantage of using the syntactic constrained context has not yet been fully exploited when yielding statistical seman- tics from word distributions. Our goal is thereof to study the salience of these syntactic depend- encies in regulating statistical semantics, which can improve the acquisition of semantic knowl- edge in the Vector Space Model (VSM). 2 Related work Padó and Lapata (2007) attempted to investigate the role of each single type of syntactic depend- ency in their syntactically conditioned VSM. They assumed a direct dependency as an undi- rected path (with a length of 1) in the graph of syntactic dependencies. In addition to this, they experimented a predefined (oblique) weighting scheme (Keenan and Comrie, 1977) in ranking dependencies, i.e. subject to verb: 5, object to verb: 4, prepositional phrase to verb: 3, etc. The optimal VSM they derived was equipped with inversely weighting dependencies within the path length less than 3, rather than this predefined scheme. Although they investigated a commonly adopted case of syntactic dependencies with the path length equal to 1, the mapping function for reducing data sparseness and dimensionality of their VSM, e.g. congregating any paths ending with the same word, has obscured distinguishing the dependences in predicting …

Content truncated to 500 words