Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment

Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM), pages 160–166 New Orleans, June 5-6, 2018. c⃝2018 Association for Computational Linguistics Integrating Multiplicative Features into Supervised Distributional Methods for Lexical Entailment Tu Vu College of Information and Computer Sciences University of Massachusetts Amherst Amherst, MA, USA tuvu@cs.umass.edu Vered Shwartz Computer Science Department Bar-Ilan University Ramat-Gan, Israel vered1986@gmail.com Abstract Supervised distributional methods are applied successfully in lexical entailment, but recent work questioned whether these methods ac- tually learn a relation between two words. Specifically, Levy et al. (2015) claimed that linear classifiers learn only separate properties of each word. We suggest a cheap and easy way to boost the performance of these meth- ods by integrating multiplicative features into commonly used representations. We provide an extensive evaluation with different classi- fiers and evaluation setups, and suggest a suit- able evaluation setup for the task, eliminating biases existing in previous ones. 1 Introduction Lexical entailment is concerned with identifying the semantic relation, if any, holding between two words, as in (pigeon, hyponym, animal). The pop- ularity of the task stems from its potential rele- vance to various NLP applications, such as ques- tion answering and recognizing textual entailment (Dagan et al., 2013) that often rely on lexical se- mantic resources with limited coverage like Word- net (Miller, 1995). Relation classifiers can be used either within applications or as an intermediate step in the construction of lexical resources which is often expensive and time-consuming. Most methods for lexical entailment are distri- butional, i.e., the semantic relation holding be- tween x and y is recognized based on their distributional vector representations. While the first methods were unsupervised and used high- dimensional sparse vectors (Weeds and Weir, 2003; Kotlerman et al., 2010; Santus et al., 2014), in recent years, supervised methods became popu- lar (Baroni et al., 2012; Roller et al., 2014; Weeds et al., 2014). These methods are mostly based on word embeddings (Mikolov et al., 2013b; Pen- nington et al., 2014a) utilizing various vector com- binations that are designed to capture relational in- formation between two words. While most previous work reported success us- ing supervised methods, some questions remain unanswered: First, several works suggested that supervised distributional methods are incapable of inferring the relationship between two words, but rather rely on independent properties of each word (Levy et al., 2015; Roller and Erk, 2016; Shwartz et al., 2016), making them sensitive to training data; Second, it remains unclear what is the most appropriate representation and classifier; previ- ous studies reported inconsistent results with Con- cat⟨ ⃗vx⊕ ⃗vy⟩ (Baroni et al., 2012) and Diff⟨⃗vy− ⃗vx⟩ (Roller et al., 2014; Weeds et al., 2014; Fu et al., 2014), using various classifiers. In this paper, we investigate the effectiveness of multiplicative features, namely, the element-wise multiplication Mult⟨ ⃗vx ⊙ ⃗vy⟩, and the squared dif- ference Sqdiff⟨(⃗vy − ⃗vx) ⊙ (⃗vy − ⃗vx)⟩. These features, similar to the cosine similarity and the Euclidean distance, might capture a different no- tion of interaction information …

Content truncated to 500 words