Quantificational features in distributional word representations

Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics (*SEM 2016), pages 1–11, Berlin, Germany, August 11-12, 2016. Quantificational features in distributional word representations Tal Linzen1,2 Emmanuel Dupoux1 Benjamin Spector2 1Laboratoire de Sciences Cognitives et Psycholinguistique 2Institut Jean Nicod ´Ecole Normale Sup´erieure PSL Research University {tal.linzen, benjamin.spector}@ens.fr emmanuel.dupoux@gmail.com Abstract Do distributional word representations en- code the linguistic regularities that theo- ries of meaning argue they should encode? We address this question in the case of the logical properties (monotonicity, force) of quantificational words such as everything (in the object domain) and always (in the time domain). Using the vector offset ap- proach to solving word analogies, we find that the skip-gram model of distributional semantics behaves in a way that is remark- ably consistent with encoding these fea- tures in some domains, with accuracy ap- proaching 100%, especially with medium- sized context windows. Accuracy in oth- ers domains was less impressive. We com- pare the performance of the model to the behavior of human participants, and find that humans performed well even where the models struggled. 1 Introduction Vector-space models of lexical semantics (VSMs) represent words as points in a high-dimensional space. Similar words are represented by points that are close together in the space. VSMs are typ- ically trained on a corpus in an unsupervised way; the goal is for words that occur in similar contexts to be assigned similar representations. The con- text of a word in a corpus is often defined as the set of words that occur in a small window around the word of interest (Lund and Burgess, 1996; Turney and Pantel, 2010). VSM representations have been shown to be useful in improving the performance of NLP systems (Turian et al., 2010; Bansal et al., 2014) as well as in predicting cognitive measures such as similarity judgments and semantic priming (Jones et al., 2006; Hill et al., 2015). While there is evidence that VSM representa- tions encode useful information about the mean- ing of open-class words such as dog or table, less is known about the extent to which they capture abstract linguistic properties, in particular the as- pects of word meaning that are crucial in logical reasoning. Some have conjectured that those prop- erties are unlikely to be encoded in VSMs (Lewis and Steedman, 2013), but evidence that VSMs en- code features such as syntactic category or verb tense suggests that this pessimism is premature (Mikolov et al., 2013c; Levy and Goldberg, 2014). The goal of this paper is to evaluate to what ex- tent logical features are encoded in VSMs. We un- dertake a detailed analysis of words with quantifi- cational features, such as everybody or nowhere. To assess whether a particular linguistic feature is encoded in a vector space, we adopt the vector off- set approach to the analogy task (Turney, 2006; Mikolov et al., 2013c; Dunbar et al., 2015). In the analogy task, a system is requested to fill in the blank in a sentence: (1) man is to …

Content truncated to 500 words