UMDuluth-BlueTeam: SVCSTS - A Multilingual and Chunk Level Semantic Similarity System

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 107–110, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics UMDuluth-BlueTeam : SVCSTS - A Multilingual and Chunk Level Semantic Similarity System Sakethram Karumuri Viswanadh Kumar Reddy Vuggumudi Sai Charan Raj Chitirala Department of Computer Science University of Minnesota Duluth {karum006, vuggu001, chiti001}@d.umn.edu Abstract This paper describes SVCSTS, a system that was submitted in SemEval-2015 Task 2: Se- mantic Textual Similarity(STS)(Agirre et al., 2015). The task has 3 subtasks viz., English STS, Spanish STS and Interpretable STS. SVCSTS uses Monolingual word aligner (Sul- tan et al., May 2014), supervised machine learning, Google and Bing translator API’s. Various runs of the system outperformed all other participating systems in Interpretable STS for non-chunked sentence input. 1 Introduction Semantic Textual Similarity gives a quantifier to evaluate semantic equivalence between two sen- tences. Earlier SemEval tasks (Agirre et al., 2012), (Agirre et al., 2013), (Agirre et al., 2014) focused on finding the semantic equivalence between sentences in English and Spanish. A new pilot task was intro- duced this year to find which parts (chunks) of the sentences are equivalent in meaning. SVCSTS is an extension to (Sultan et al., 2014) and it handles both Spanish STS and Interpretable STS. SVCSTS uses Monolingual word aligner (Sul- tan et al., May 2014), supervised machine learning techniques, Google and Bing translator API’s. Section 2 describes a brief overview of SVCSTS’s approach for various subtasks. Section 3 outlines the performance of SVCSTS in various subtasks of SemEval 2015 Task-2. 2 System Description Following 3 sub sections describe SVCSTS’s ap- proach for the 3 subtasks. 2.1 English STS This task was about finding the semantic similarity between English sentences. (Sultan et al., 2014) sys- tem was used to find the semantic equivalence be- tween two sentences and a score on a scale of 0-5 was given. 2.2 Spanish STS Spanish STS is built upon English STS to calculate similarity scores for a given pair of Spanish sen- tences on a scale of 0 to 4. Spanish sentences were translated to English, fed to English STS system and the scores are scaled accordingly. Translations were done using Bing Translator API (Bing Transla- tor API) and Google Translate API. Two translators were used to improve the accuracy of the transla- tions. Google Translate API was obtained from (Kashyap et al., 2014). We used this system to get multiple translations of each chunk in a sentence. Multiple sentences are generated by combining the top two translations of each chunk. We then randomly pick a maximum of ten sentences for each Spanish sentence. Translation pairs are formed by choosing corresponding numbered sentences from sentence 1 and sentence 2 translations. We limited the number of translations to 10 to reduce the overall computation time. Translation pairs were then passed to English STS system. Final score was obtained as the average 107 taken from all translation pairs for a given Spanish sentence pair and the score is scaled accordingly. 2.3 Interpretable STS …

Content truncated to 500 words