DCU: Using Distributional Semantics and Domain Adaptation for the Semantic Textual Similarity SemEval-2015 Task 2

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 143–147, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics DCU: Using Distributional Semantics and Domain Adaptation for the Semantic Textual Similarity SemEval-2015 Task 2 Piyush Arora, Chris Hokamp, Jennifer Foster, Gareth J.F.Jones ADAPT Centre School of Computing Dublin City University Dublin, Ireland {parora,chokamp,jfoster,gjones}@computing.dcu.ie Abstract We describe the work carried out by the DCU team on the Semantic Textual Similarity task at SemEval-2015. We learn a regression model to predict a semantic similarity score between a sentence pair. Our system exploits distributional semantics in combination with tried-and-tested features from previous tasks in order to compute sentence similarity. Our team submitted 3 runs for each of the five En- glish test sets. For two of the test sets, belief and headlines, our best system ranked second and fourth out of the 73 submitted systems. Our best submission averaged over all test sets ranked 26 out of the 73 systems. 1 Introduction This paper describes DCU’s participation in the Se- mEval 2015 English Semantic Textual Similarity (STS) task, whose goal is to predict how similar in meaning two sentences are (Agirre et al., 2014). The semantic similarity between two sentences is defined on a scale from 0 (no relation) to 5 (semantic equivalence). Thus, given a sentence pair, our aim is to learn a model which outputs a score between 0 and 5 reflecting the semantic similarity between the two sentences. We explore distributional representations of words computed using neural networks – specifi- cally Word2Vec vectors (Mikolov et al., 2013) – and we design features which attempt to encode seman- tic similarity at the sentence level. We also experi- ment with several methods of data selection, both for training word embeddings, and for selecting training data for our regression models. We submitted three runs for this task: for all three runs, the features used are identical, and the only difference between them is the training instance selection method used. 2 Data and Resources The training data for the task is comprised of all the corpora from previous years STS tasks: STS- 2012, STS-2013 and STS-2014 (Agirre et al., 2012; Agirre et al., 2013; Agirre et al., 2014). The test data is taken from five domains: answers-forums, answers-students, belief, headlines and images. Two domains (headlines and images) have some training data available from the previous STS tasks1 – the other three have been introduced for the first time. We use the Word2Vec (W2V) representation for computing semantic similarity between two words. We then expand to incorporate the similarity be- tween two sentences. Using W2V, a word can be represented as a vector of D dimensions, with each dimension capturing some aspect of the word’s meaning in the form of different concepts learnt from the trained model. We use the gensim W2V implementation ( ˇReh˚uˇrek and Sojka, 2010). We use the text8 Wikipedia corpus to train our general W2V model. This corpus is comprised of 100MB of compressed Wikipedia data.2 We …

Content truncated to 500 words