Relations such as Hypernymy: Identifying and Exploiting Hearst Patterns in Distributional Vectors for Lexical Entailment

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2163–2172, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Relations such as Hypernymy: Identifying and Exploiting Hearst Patterns in Distributional Vectors for Lexical Entailment Stephen Roller Department of Computer Science The University of Texas at Austin roller@cs.utexas.edu Katrin Erk Department of Linguistics The University of Texas at Austin katrin.erk@mail.utexas.edu Abstract We consider the task of predicting lexical entailment using distributional vectors. We perform a novel qualitative analysis of one existing model which was previously shown to only measure the prototypicality of word pairs. We find that the model strongly learns to identify hypernyms using Hearst patterns, which are well known to be predictive of lexi- cal relations. We present a novel model which exploits this behavior as a method of fea- ture extraction in an iterative procedure sim- ilar to Principal Component Analysis. Our model combines the extracted features with the strengths of other proposed models in the literature, and matches or outperforms prior work on multiple data sets. 1 Introduction As the field of Natural Language Processing has de- veloped, more ambitious semantic tasks are starting to be addressed, such as Question Answering (QA) and Recognizing Textual Entailment (RTE). These systems often depend on the use of lexical resources like WordNet in order to infer entailments for indi- vidual words, but these resources are expensive to develop, and always have limited coverage. To address these issues, many works have con- sidered on how lexical entailments can be derived automatically using distributional semantics. Some focus mostly on the use of unsupervised techniques, and study measures which emphasize particular word relations (Baroni and Lenci, 2011). Many are based on the Distributional Inclusion Hypothesis, which states that the contexts in which a hypernym appears are a superset of its hyponyms’ contexts (Zhitomirsky-Geffet and Dagan, 2005; Kotlerman et al., 2010). More recently, a great deal of work has pushed toward using supervised methods (Baroni et al., 2012; Roller et al., 2014; Weeds et al., 2014; Levy et al., 2015; Kruszewski et al., 2015), varying by their experimental setup or proposed model. Yet the literature disagrees about which models are strongest (Weeds et al., 2014; Roller et al., 2014), or even if they work at all (Levy et al., 2015). Indeed, Levy et al. (2015) showed that two exist- ing lexical entailment models fail to account for similarity between the antecedent and consequent, and conclude that such models are only learning to predict prototypicality: that is, they predict that cat entails animal because animal is usually en- tailed, and therefore will also predict that sofa en- tails animal. Yet it remains unclear why such models make for such strong baselines (Weeds et al., 2014; Kruszewski et al., 2015; Levy et al., 2015). We present a novel qualitative analysis of one pro- totypicality classifier, giving new insight into why prototypicality classifiers perform strongly in the lit- erature. We find the model overwhelmingly learns to identify hypernyms using Hearst patterns avail- …

Content truncated to 500 words