Learning Adjective Meanings with a Tensor-Based Skip-Gram Model

Proceedings of the 19th Conference on Computational Language Learning, pages 327–331, Beijing, China, July 30-31, 2015. c⃝2015 Association for Computational Linguistics Learning Adjective Meanings with a Tensor-Based Skip-Gram Model Jean Maillard University of Cambridge Computer Laboratory jean@maillard.it Stephen Clark University of Cambridge Computer Laboratory sc609@cam.ac.uk Abstract We present a compositional distributional semantic model which is an implementa- tion of the tensor-based framework of Co- ecke et al. (2011). It is an extended skip- gram model (Mikolov et al., 2013) which we apply to adjective-noun combinations, learning nouns as vectors and adjectives as matrices. We also propose a novel measure of adjective similarity, and show that adjective matrix representations lead to improved performance in adjective and adjective-noun similarity tasks, as well as in the detection of semantically anomalous adjective-noun pairs. 1 Introduction A number of approaches have emerged for com- bining compositional and distributional seman- tics. Some approaches assume that all words and phrases are represented by vectors living in the same semantic space, and use mathematical op- erations such as vector addition and element-wise multiplication to combine the constituent vectors (Mitchell and Lapata, 2008). In these relatively simple methods, the composition function does not typically depend on its arguments or their syn- tactic role in the sentence. An alternative which makes more use of gram- matical structure is the recursive neural network approach of Socher et al. (2010). Constituent vec- tors in a phrase are combined using a matrix and non-linearity, with the resulting vector living in the same vector space as the inputs. The matrices can be parameterised by the syntactic type of the com- bining words or phrases (Socher et al., 2013; Her- mann and Blunsom, 2013). Socher et al. (2012) extend this idea by representing the meanings of words and phrases as both a vector and a matrix, introducing a form of lexicalisation into the model. A further extension, which moves us closer to formal semantics (Dowty et al., 1981), is to build a semantic representation in step with the syntac- tic derivation, and have the embeddings of words be determined by their syntactic type. Coecke et al. (2011) achieve this by treating relational words such as verbs and adjectives as functions in the semantic space. The functions are assumed to be multilinear maps, and are therefore realised as tensors, with composition being achieved through tensor contraction.1 While the framework speci- fies the “shape” or semantic type of these tensors, it makes no assumption about how the values of these tensors should be interpreted (nor how they can be learned). A proposal for the case of adjective-noun com- binations is given by Baroni and Zamparelli (2010) (and also Guevara (2010)). Their model represents adjectives as matrices over noun space, trained via linear regression to approximate the “holistic” adjective-noun vectors from the corpus. In this paper we propose a new solution to the problem of learning adjective meaning represen- tations. The model is an implementation of the tensor framework of Coecke et al. (2011), here applied to adjective-noun …

Content truncated to 500 words