Deriving Boolean structures from distributional vectors

Deriving Boolean structures from distributional vectors German Kruszewski Denis Paperno Center for Mind/Brain Sciences University of Trento firstname.lastname@unitn.it Marco Baroni Abstract Corpus-based distributional semantic mod- els capture degrees of semantic relatedness among the words of very large vocabular- ies, but have problems with logical phe- nomena such as entailment, that are in- stead elegantly handled by model-theoretic approaches, which, in turn, do not scale up. We combine the advantages of the two views by inducing a mapping from distributional vectors of words (or sentences) into a Boolean structure of the kind in which natural lan- guage terms are assumed to denote. We evaluate this Boolean Distributional Seman- tic Model (BDSM) on recognizing entailment between words and sentences. The method achieves results comparable to a state-of-the- art SVM, degrades more gracefully when less training data are available and displays inter- esting qualitative properties. 1 Introduction Different aspects of natural language semantics have been studied from different perspectives. Distribu- tional semantic models (Turney and Pantel, 2010) induce large-scale vector-based lexical semantic representations from statistical patterns of word us- age. These models have proven successful in tasks relying on meaning relatedness, such as synonymy detection (Landauer and Dumais, 1997), word sense discrimination (Sch¨utze, 1997), or even measuring phrase plausibility (Vecchi et al., 2011). On the other hand, logical relations and operations, such as entailment, contradiction, conjunction and nega- tion, receive an elegant treatment in formal seman- tic models. The latter lack, however, general pro- cedures to learn from data, and consequently have problems scaling up to real-life problems. Formal semantics captures fundamental aspects of meaning in set-theoretic terms: Entailment, for example, is captured as the inclusion relation be- tween the sets (of the relevant type) denoted by words or other linguistic expressions, e.g., sets of possible worlds that two propositions hold of (Chier- chia and McConnell-Ginet, 2000, 299). In finite models, a mathematically convenient way to repre- sent these denotations is to encode them in Boolean vectors, i.e., vectors of 0s and 1s (Sasao, 1999, 21). Given all elements ei in the domain in which linguis- tic expressions of a certain type denote, the Boolean vector associated to a linguistic expression of that type has 1 in position i if ei ∈ S for S the set de- noted by the expression, 0 otherwise. An expression a entailing b will have a Boolean vector including the one of b, in the sense that all positions occupied by 1s in the b vector are also set to 1 in the a vec- tor. Very general expressions (entailing nearly ev- erything else) will have very dense vectors, whereas very specific expressions will have very sparse vec- tors. The negation of an expression a will denote a “flipped” version of the a Boolean vector. Vice versa, two expressions with at least partially com- patible meanings will have some overlap of the 1s in their vectors; conjunction and disjunction are carried through with the obvious bit-wise operations, etc. To narrow the gap between the large-scale induc- tive …

Content truncated to 500 words