UCAM-CORE: Incorporating structured distributional similarity into STS

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 85–89, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics UCAM-CORE: Incorporating structured distributional similarity into STS Tamara Polajnar Laura Rimell Douwe Kiela Computer Laboratory University of Cambridge Cambridge CB3 0FD, UK {tamara.polajnar,laura.rimell,douwe.kiela}@cl.cam.ac.uk Abstract This paper describes methods that were sub- mitted as part of the *SEM shared task on Semantic Textual Similarity. Multiple kernels provide different views of syntactic structure, from both tree and dependency parses. The kernels are then combined with simple lex- ical features using Gaussian process regres- sion, which is trained on different subsets of training data for each run. We found that the simplest combination has the highest consis- tency across the different data sets, while in- troduction of more training data and models requires training and test data with matching qualities. 1 Introduction The Semantic Textual Similarity (STS) shared task consists of several data sets of paired passages of text. The aim is to predict the similarity that hu- man annotators have assigned to these aligned pairs. Text length and grammatical quality vary between the data sets, so our submissions to the task aimed to investigate whether models that incorporate syntac- tic structure in similarity calculation can be consis- tently applied to diverse and noisy data. We model the problem as a combination of ker- nels (Shawe-Taylor and Cristianini, 2004), each of which calculates similarity based on a different view of the text. State-of-the-art results on text classifi- cation have been achieved with kernel-based classi- fication algorithms, such as the support vector ma- chine (SVM) (Joachims, 1998), and the methods here can be adapted for use in multiple kernel classi- fication, as in Polajnar et al. (2011). The kernels are combined using Gaussian process regression (GPR) (Rasmussen and Williams, 2006). It is important to note that the combination strategy described here is only a different way of viewing the regression- combined mixture of similarity measures approach that is already popular in STS systems, including several that participated in previous SemEval tasks (Croce et al., 2012; B¨ar et al., 2012). Likewise, oth- ers, such as Croce et al. (2012), have used tree and dependency parse information as part of their sys- tems; however, we use a tree kernel approach based on a novel encoding method introduced by Zanzotto et al. (2011) and from there derive two dependency- based methods. In the rest of this paper we will describe our sys- tem, which consists of distributional similarity (Sec- tion 2.1), several kernel measures (Section 2.2), and a combination method (Section 2.3). This will be followed by the description of our three submissions (Section 3), and a discussion of the results (Sec- tion 4). 2 Methods At the core of all the kernel methods is either sur- face, distributional, or syntactic similarity between sentence constituents. The methods themselves en- code sentences into vectors or sets of vectors, while the similarity between any two vectors is …

Content truncated to 500 words