A Vector Space for Distributional Semantics for Entailment

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 2052–2062, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A Vector Space for Distributional Semantics for Entailment ∗ James Henderson and Diana Nicoleta Popa Xerox Research Centre Europe james.henderson@xrce.xerox.com and diana.popa@xrce.xerox.com Abstract Distributional semantics creates vector- space representations that capture many forms of semantic similarity, but their re- lation to semantic entailment has been less clear. We propose a vector-space model which provides a formal foundation for a distributional semantics of entailment. Us- ing a mean-field approximation, we de- velop approximate inference procedures and entailment operators over vectors of probabilities of features being known (ver- sus unknown). We use this framework to reinterpret an existing distributional- semantic model (Word2Vec) as approxi- mating an entailment-based model of the distributions of words in contexts, thereby predicting lexical entailment relations. In both unsupervised and semi-supervised experiments on hyponymy detection, we get substantial improvements over previ- ous results. 1 Introduction Modelling entailment is a fundamental issue in computational semantics. It is also important for many applications, for example to produce ab- stract summaries or to answer questions from text, where we need to ensure that the input text entails the output text. There has been a lot of interest in modelling entailment in a vector-space, but most of this work takes an empirical, often ad-hoc, ap- proach to this problem, and achieving good results has been difficult (Levy et al., 2015). In this work, we propose a new framework for modelling entail- ment in a vector-space, and illustrate its effective- ∗This work was partially supported by French ANR grant CIFRE N 1324/2014. ⇒ unk f g ¬f unk 1 0 0 0 f 1 1 0 0 g 1 0 1 0 ¬f 1 0 0 1 Table 1: Pattern of logical entailment between nothing known (unk), two different features f and g known, and the complement of f (¬f) known. ness with a distributional-semantic model of hy- ponymy detection. Unlike previous vector-space models of entail- ment, the proposed framework explicitly models what information is unknown. This is a crucial property, because entailment reflects what infor- mation is and is not known; a representation y en- tails a representation x if and only if everything that is known given x is also known given y. Thus, we model entailment in a vector space where each dimension represents something we might know. As illustrated in Table 1, knowing that a feature f is true always entails knowing that same feature, but never entails knowing that a different feature g is true. Also, knowing that a feature is true always entails not knowing anything (unk), since strictly less information is still entailment, but the reverse is never true. Table 1 also illustrates that knowing that a feature f is false (¬f) patterns exactly the same way as knowing that an unrelated feature g is true. This illustrates that the relevant dichotomy for entailment is known versus unknown, and not true versus …

Content truncated to 500 words