Dissecting the Practical Lexical Function Model for Compositional Distributional Semantics

Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (*SEM 2015), pages 153–158, Denver, Colorado, June 4–5, 2015. Dissecting the Practical Lexical Function Model for Compositional Distributional Semantics Abhijeet Gupta, Jason Utt and Sebastian Pad´o Institut f¨ur Maschinelle Sprachverarbeitung Universit¨at Stuttgart [guptaat|uttjn|pado]@ims.uni-stuttgart.de Abstract The Practical Lexical Function model (PLF) is a recently proposed compositional distribu- tional semantic model which provides an el- egant account of composition, striking a bal- ance between expressiveness and robustness and performing at the state-of-the-art. In this paper, we identify an inconsistency in PLF be- tween the objective function at training and the prediction at testing which leads to an over- counting of the predicate’s contribution to the meaning of the phrase. We investigate two pos- sible solutions of which one (the exclusion of simple lexical vector at test time) improves per- formance significantly on two out of the three composition datasets. 1 Introduction Compositional distributional semantic models (CDSMs) make an important theoretical contribution, explaining the meaning of a phrase by the meanings of its parts. They have also found application in psycholinguistics (Lenci, 2011), in sentiment analysis (Socher et al., 2012), and in machine translation (Kalchbrenner and Blunsom, 2013). A first generation of CDSMs represented all words as vectors and combined them by component-wise operations (Mitchell and Lapata, 2010). Given the conceptual limitations of this simple approach, nu- merous models were subsequently proposed which represent the meaning of predicates as higher-order algebraic objects such as matrices and tensors (Ba- roni and Zamparelli, 2010; Guevara, 2010; Coecke et al., 2010). For example, one-place predicates such as adjectives or intransitive verbs can be modeled as matrices (order-2 tensors), and two-place predicates, e.g., transitive verbs, as order-3 tensors, and so forth. While such tensors enable mathematically elegant ac- counts of composition, their large degrees of freedom lead to severe sparsity issues when they are learned from corpora. The recently proposed Practical Lexical Function model (PLF; Paperno et al., 2014) represents a com- promise between these two extremes by restricting itself to vectors and matrices, effectively reducing sparsity while retaining state-of-the-art performance across multiple datasets. It does away with tensors by ignoring interactions among the arguments of predi- cates p. Instead, each argument position arg is mod- eled as a matrix □arg p that is applied to a vector for the argument’s meaning, −→a . The meaning of the phrase is then defined as the sum of the lexical meaning of the predicate, −→p , and the contributions of each ar- gument (see Fig. 1). The matrices can be learned in a supervised manner with regression from pairs of corpus-extracted vectors for arguments and phrases. In this paper, we identify an inconsistency between the training and testing phases of the PLF. More specifically, we show that its composition procedure leads to over-counting of the contribution of the pred- icate. We propose two remedies to harmonize the training and prediction phases – by excluding the predicate meaning from either training or testing. In an evaluation of the standard …

Content truncated to 500 words