DCU-SEManiacs at SemEval-2016 Task 1: Synthetic Paragram Embeddings for Semantic Textual Similarity

Proceedings of SemEval-2016, pages 656–662, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics DCU-SEManiacs at SemEval-2016 Task 1: Synthetic Paragram Embeddings for Semantic Textual Similarity Chris Hokamp, Piyush Arora ADAPT Centre School of Computing Dublin City University Dublin, Ireland {chokamp,parora}@computing.dcu.ie Abstract We experiment with learning word representa- tions designed to be combined into sentence- level semantic representations, using an ob- jective function which does not directly make use of the supervised scores provided with the training data, instead opting for a simpler objective which encourages similar phrases to be close together in the embedding space. This simple objective lets us start with high- quality embeddings trained using the Para- phrase Database (PPDB) (Wieting et al., 2015; Ganitkevitch et al., 2013), and then tune these embeddings using the official STS task train- ing data, as well as synthetic paraphrases for each test dataset, obtained by pivoting through machine translation. Our submissions include runs which only compare the similarity of phrases in the em- bedding space, directly using the similarity score to produce predictions, as well as a run which uses vector similarity in addition to a suite of features we investigated for our 2015 Semeval submission. For the crosslingual task, we simply translate the Spanish sentences to English, and use the same system we designed for the monolingual task. 1 Introduction We describe the work carried out by the DCU- SEManiacs team on the Semantic Textual Similar- ity (STS) task at SemEval-2016 (Agirre et al., 2013; Agirre et al., 2014; Nakov et al., 2015). The main ideas we investigate in our systems are: 1. Using a margin-based objective function to train high-quality sentence embeddings with- out using supervised scores 2. Creating new synthetic training data using ma- chine translation to generate artificial para- phrases 3. Using ensemble models to combine features generated by our embedding networks with features obtained from other sources 1.1 Task Description The Semeval Semantic Textual Similarity (STS) task provides participants with training data consisting of pairs of sentences annotated with gold-standard se- mantic similarity scores. The crowd-sourced simi- larity scores are given on a scale from 0 (no relation) to 5 (semantic equivalence). Thus, our aim is to use the training data to learn a model which predicts a score between 0 and 5 for unseen input pairs (Nakov et al., 2015). The monolingual STS task has been or- ganized each year since 2012, and most approaches have viewed the learning task as a regression prob- lem, where real-valued model output is clipped to be 0 <= ˆy <= 5 (Agirre et al., 2013; Agirre et al., 2014; Nakov et al., 2015). For two of our three STS systems, we take a novel approach to this task, and directly use the similar- ity scores produced by the embedding networks as the predicted score. When training the embedding networks, we use the gold scores only to reduce the task-internal data to segments with a high-semantic similarity – embeddings are then learned using a simplified training …

Content truncated to 500 words