Distributional Inclusion Hypothesis for Tensor-based Composition

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2849–2860, Osaka, Japan, December 11-17 2016. Distributional Inclusion Hypothesis for Tensor-based Composition Dimitri Kartsaklis and Mehrnoosh Sadrzadeh Queen Mary University of London School of Electronic Engineering and Computer Science Mile End Road, London, UK {d.kartsaklis;mehrnoosh.sadrzadeh}@qmul.ac.uk Abstract According to the distributional inclusion hypothesis, entailment between words can be measured via the feature inclusions of their distributional vectors. In recent work, we showed how this hypothesis can be extended from words to phrases and sentences in the setting of compositional distributional semantics. This paper focuses on inclusion properties of tensors; its main con- tribution is a theoretical and experimental analysis of how feature inclusion works in different concrete models of verb tensors. We present results for relational, Frobenius, projective, and holistic methods and compare them to the simple vector addition, multiplication, min, and max models. The degrees of entailment thus obtained are evaluated via a variety of existing word- based measures, such as Weed’s and Clarke’s, KL-divergence, APinc, balAPinc, and two of our previously proposed metrics at the phrase/sentence level. We perform experiments on three en- tailment datasets, investigating which version of tensor-based composition achieves the highest performance when combined with the sentence-level measures. 1 Introduction Distributional hypothesis asserts that words that often occur in the same contexts have similar meanings (Firth, 1957). Naturally these models are used extensively to measure the semantic similarity of words (Turney and Pantel, 2010). Similarity is an a-directional relationship and computational linguists are also interested in measuring degrees of directional relationships between words. Distributional inclusion hypothesis is exactly about such relationships, and particularly, about entailment (Dagan et al., 1999; Geffet and Dagan, 2005; Herbelot and Ganesalingam, 2013). It states that a word u entails a word v if in any context that word u is used so can be word v. For example, in a corpus of sentences ‘a boy runs’, ‘a person runs’, ‘a person sleeps’, according to this hypothesis, boy ⊢ person, since wherever ‘boy’ is used, so is ‘person’. Formally, we have that u entails v if features of u are included in features of v, where features are non-zero contexts. In this example, boy ⊢ person, since {run} ⊂ {run, sleep}. For the same reasons that the distributional hypothesis is not directly applicable to phrases and sen- tences, the distributional inclusion hypothesis does not scale up from words to larger language con- stituents. In a nutshell, this is because the majority of phrases and sentences of language do not frequently occur in corpora of text, thus reliable statistics cannot be collected for them. In distributional models, this problem is addressed with the provision of composition operators, the purpose of which is to combine the statistics of words and obtain statistics for phrases and sentences. In recent work, we have applied the same compositionality principles to lift the entailment relation from word level to phrase/sentence level (Kartsaklis and Sadrzadeh, 2016; Balkır et al., 2016a; Balkır et al., 2016b; Balkır, …

Content truncated to 500 words