Improving the Lexical Function Composition Model with Pathwise Optimized Elastic-Net Regression

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 434–442, Gothenburg, Sweden, April 26-30 2014. c⃝2014 Association for Computational Linguistics Improving the Lexical Function Composition Model with Pathwise Optimized Elastic-Net Regression Jiming Li and Marco Baroni and Georgiana Dinu Center for Mind/Brain Sciences University of Trento, Italy (jiming.li|marco.baroni|georgiana.dinu)@unitn.it Abstract In this paper, we show that the lexical function model for composition of dis- tributional semantic vectors can be im- proved by adopting a more advanced re- gression technique. We use the pathwise coordinate-descent optimized elastic-net regression method to estimate the compo- sition parameters, and compare the result- ing model with several recent alternative approaches in the task of composing sim- ple intransitive sentences, adjective-noun phrases and determiner phrases. Experi- mental results demonstrate that the lexical function model estimated by elastic-net re- gression achieves better performance, and it provides good qualitative interpretabil- ity through sparsity constraints on model parameters. 1 Introduction Vector-based distributional semantic models of word meaning have gained increased attention in recent years (Turney and Pantel, 2010). Differ- ent from formal semantics, distributional seman- tics represents word meanings as vectors in a high- dimensional semantic space, where the dimen- sions are given by co-occurring contextual fea- tures. The intuition behind these models lies in the fact that words which are similar in meaning often occur in similar contexts, e.g., moon and star might both occur with sky, night and bright. This leads to convenient ways to measure similar- ity between different words using geometric meth- ods (e.g., the cosine of the angle between two vectors that summarize their contextual distribu- tion). Distributional semantic models have been successfully applied to many tasks in linguistics and cognitive science (Griffiths et al., 2007; Foltz et al., 1998; Laham, 1997; McDonald and Brew, 2004). However, most of these tasks only deal with isolated words, and there is a strong need to construct representations for longer linguistic structures such as phrases and sentences. In or- der to achieve this goal, the principle of com- positionality of linguistic structures, which states that complex linguistic structures can be formed through composition of simple elements, is ap- plied to distributional vectors. Therefore, in recent years, the problem of composition within distribu- tional models has caught many researchers’ atten- tion (Clark, 2013; Erk, 2012). A number of compositional frameworks have been proposed and tested. Mitchell and Lapata (2008) propose a set of simple component-wise operations, such as multiplication and addition. Later, Guevara (2010) and Baroni and Zampar- elli (2010) proposed more elaborate methods, in which composition is modeled as matrix-vector multiplication operations. Particularly new to their approach is the proposal to estimate model param- eters by minimizing the distance of the composed vectors to corpus-observed phrase vectors. For ex- ample, Baroni and Zamparelli (2010) consider the case of Adjective-Noun composition and model it as matrix-vector multiplication: adjective matrices are parameters to be estimated and nouns are co- occurrence vectors. The model parameter estima- tion procedure becomes a multiple response mul- tivariate …

Content truncated to 500 words