UMDeep at SemEval-2017 Task 1: End-to-End Shared Weight LSTM Model for Semantic Textual Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 180–184, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics UMDeep at SemEval-2017 Task 1: End-to-End Shared Weight LSTM Model for Semantic Textual Similarity Joe Barrow University of Maryland Computer Science jdbarrow@cs.umd.edu Denis Peskov University of Maryland Computer Science dpeskov@cs.umd.edu Abstract We describe a modified shared-LSTM net- work for the Semantic Textual Similar- ity (STS) task at SemEval-2017. The network builds on previously explored Siamese network architectures. We treat max sentence length as an additional hy- perparameter to be tuned (beyond learn- ing rate, regularization, and dropout). Our results demonstrate that hand-tuning max sentence training length significantly im- proves final accuracy. After optimiz- ing hyperparameters, we train the net- work on the multilingual semantic similar- ity task using pre-translated sentences. We achieved a correlation of 0.4792 for all the subtasks. We achieved the fourth highest team correlation for Task 4b, which was our best relative placement. 1 Introduction Semantic Textual Similarity (STS) has been a sta- ple of the SemEval competition and requires sys- tems that automatically identify the semantic re- latedness of two sentences. The resulting sys- tem could be used down-stream in many impor- tant NLP tasks, such as scoring the output of a machine translation system or finding related doc- ument/query pairs in web search. The data available for this competition has been updated annually and contains gold-label, human- evaluated scores based on sentence pairs across multiple languages ((Agirre et al., 2012), (Agirre et al., 2013), (Agirre et al., 2014), (Agirre et al., 2015)). The gold label for each sentence pair is in the range [0, 5], with 0 being the sentences are completely dissimilar to 5 being the sentences are completely equivalent. (Agirre et al., 2016) The task is not restricted to English or mono- lingual similarity scoring. The 2017 SemEval task consists of seven different tracks, each with a different language pair: Arabic-Arabic, Arabic-English, Spanish-Spanish, Spanish- English, an additional Spanish-English track, English-English, and English-Turkish. We avoid language-specific feature engineering and take a representation learning approach to STS. This requires constructing directly-comprable sentence representations that can be induced from the limited amounts of annotated STS training data. We present a modified version of the Siamese Long Short-Term Memory (LSTM) network to solve this problem. (Mueller and Thyagarajan, 2016) A Siamese network is one in which parame- ters between layers are shared, and are updated in parallel during the learning phase. For the seman- tic relatedness task, this allows two sentences to be encoded into the same space using a single shared recurrent neural network. The dual-encoding en- ables the use of end-to-end supervised deep learn- ing, using only the surface forms of the sentences and the gold labels. We extend the Siamese LSTM in two ways. First, we consider the semantic relatedness as a classification, rather than a regression problem. Initially, semantic relatedness appears to be a con- tinuous one-dimensional measure suitable for re- gression. However, there are many subtleties within the …

Content truncated to 500 words