NUIG-UNLP at SemEval-2016 Task 1: Soft Alignment and Deep Learning for Semantic Textual Similarity

Proceedings of SemEval-2016, pages 712–717, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics NUIG-UNLP at SemEval-2016 Task 1: Soft Alignment and Deep Learning for Semantic Textual Similarity John P. McCrae and Kartik Asooja and Nitish Aggarwal and Paul Buitelaar Insight Centre for Data Analytics National University of Ireland, Galway IDA Business Park Galway, Ireland {john.mccrae, katrik.asooja, nitish.aggarwal, paul.buitelaar}@insight-centre.org Abstract We present a multi-feature system for comput- ing the semantic similarity between two sen- tences. We introduce the use of soft align- ment for computing text similarity, and also evaluate different methods to produce it. The main features used by our system are based on alignment and Explicit Semantic Analysis. Our system was above the median scores for 4 out of the 5 datasets at SemEval 2016 STS Task 1. 1 Introduction Semantic textual similarity is the task of deciding if two sentences express a similar or identical mean- ing and requires a deep understanding of a sentence and its meaning in order to achieve high perfor- mance. Recent successful approaches to this prob- lem have been based on the idea of creating mono- lingual alignments (Sultan et al., 2014a) indicating which words in each of the two sentences corre- spond to each other. This is quite successful in many cases where many words have the same lemma, however when synonymous and semantically sim- ilar terms are used, it is much harder to construct alignment. For this reason, we propose the use of soft alignments, where instead of producing a hard linking between individual words in the sentence, we instead produce a score indicating how likely one word in a sentence is to be aligned to another word in the other sentence. We examine several methods that can be used to learn these alignments including word embeddings (Mikolov et al., 2013; Penning- ton et al., 2014) and models based on deep learn- ing that have been suggested for machine transla- tion (Bahdanau et al., 2014; Cho et al., 2014). In addition, we look into recent models for sentence and document similarity that can leverage the large amount of loosely aligned text in particular those based on Explicit Semantic Analysis (Gabrilovich and Markovitch, 2007) and recent extensions aimed at generating orthogonal representations (McCrae et al., 2013; Aggarwal et al., 2015). While these novel techniques alone can achieve high performance on the task, we note that simple metrics such as the number of overlapping terms can produce reason- able performance. For added robustness we combine features based on simple metrics with novel methods explored in this work as a multi-feature regression problem, which we solve by means of an M5 Deci- sion tree (Wang and Witten, 1996; Quinlan, 1992). The rest of the paper is structured as follows: we present our system in Section 2. We then present both our internal evaluation results and the official Task 1 results in Section 3 and finally we conclude in Section 4. 2 Methods For convenience we assume that semantic textual similarity consists …

Content truncated to 500 words