Towards Syntax-aware Compositional Distributional Semantic Models

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 721–730, Dublin, Ireland, August 23-29 2014. Towards Syntax-aware Compositional Distributional Semantic Models Lorenzo Ferrone Department of Enterprise Engineering University of Rome “Tor Vergata” Via del Politecnico, 1 00173 Roma lorenzo.ferrone@gmail.com Fabio Massimo Zanzotto Department of Enterprise Engineering University of Rome “Tor Vergata” Via del Politecnico, 1 00173 Roma fabio.massimo.zanzotto@uniroma2.it Abstract Compositional Distributional Semantics Models (CDSMs) are traditionally seen as an entire dif- ferent world with respect to Tree Kernels (TKs). In this paper, we show that under a suitable regime these two approaches can be regarded as the same and, thus, structural information and distributional semantics can successfully cooperate in CSDMs for NLP tasks. Leveraging on distributed trees, we present a novel class of CDSMs that encode both structure and distribu- tional meaning: the distributed smoothed trees (DSTs). By using DSTs to compute the similarity among sentences, we implicitly define the distributed smoothed tree kernels (DSTKs). Exper- iment with our DSTs show that DSTKs approximate the corresponding smoothed tree kernels (STKs). Thus, DSTs encode both structural and distributional semantics of text fragments as STKs do. Experiments on RTE and STS show that distributional semantics encoded in DSTKs increase performance over structure-only kernels. 1 Introduction Compositional distributional semantics is a flourishing research area that leverages distributional seman- tics (see Turney and Pantel (2010), Baroni and Lenci (2010)) to produce meaning of simple phrases and full sentences (hereafter called text fragments). The aim is to scale up the success of word-level relatedness detection to longer fragments of text. Determining similarity or relatedness among sentences is useful for many applications, such as multi-document summarization, recognizing textual entailment (Dagan et al., 2013), and semantic textual similarity detection (Agirre et al., 2013). Compositional distributional semantics models (CDSMs) are functions mapping text fragments to vectors (or higher-order tensors). Functions for simple phrases directly map distributional vectors of words to distributional vectors for the phrases (Mitchell and Lapata, 2008; Baroni and Zamparelli, 2010; Clark et al., 2008; Grefenstette and Sadrzadeh, 2011; Zanzotto et al., 2010). Functions for full sentences are generally defined as recursive functions over the ones for phrases (Socher et al., 2011; Socher et al., 2012; Kalchbrenner and Blunsom, 2013). Distributional vectors for text fragments are then used as inner layers in neural networks, or to compute similarity among text fragments via dot product. CDSMs generally exploit structured representations tx of text fragments x to derive their meaning f(tx), but the structural information, although extremely important, is obfuscated in the final vectors. Structure and meaning can interact in unexpected ways when computing cosine similarity (or dot prod- uct) between vectors of two text fragments, as shown for full additive models in (Ferrone and Zanzotto, 2013). Smoothed tree kernels (STK) (Mehdad et al., 2010; Croce et al., 2011) instead realize a clearer interaction between structural information and distributional meaning. STKs are specific realizations of convolution kernels (Haussler, 1999) where the similarity function is recursively (and, thus, composition- ally) computed. Distributional vectors are …

Content truncated to 500 words