IHS-RD-Belarus at SemEval-2016 Task 1: Multistage Approach for Measuring Semantic Similarity

Proceedings of SemEval-2016, pages 696–701, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics IHS-RD-Belarus at SemEval-2016 Task 1: Multistage Approach for Measuring Semantic Similarity Maryna Beliuha, Maryna Chernyshevich IHS Inc. / IHS Global Belarus 131 Starovilenskaya St 220123, Minsk, Belarus {Marina.Beliuga}@ihs.com, {Marina.Chernyshevich}@ihs.com Abstract This paper describes the system for rating the degree of semantic equivalence between two text snippets developed by IHS-RD-Belarus for the SemEval 2016 STS shared task (Task 1). To predict the human ratings of text simi- larity we use a support vector regression mod- el with multiple features representing similari- ty and difference scores calculated for each pair of sentences. 1 Introduction Measuring semantic equivalence between two texts has become an emerging research subject in recent years. Graded textual similarity notion can be ap- plied to a wide range of NLP tasks such as para- phrase recognition, automatic machine translation evaluation, question answering, text summariza- tion, information retrieval, etc. The SemEval STS shared task has been held an- nually since 2012 (Agirre et al., 2012; Agirre et al., 2013; Agirre et al., 2014, Agirre et al., 2015) at- tracting numerous participating teams with various approaches, such as alignment of related content word sequences (Sultan et al., 2015), measuring similarity between vector representations of texts and using machine learning algorithms for compu- ting multiple lexical, syntactic and semantic fea- tures. In this article, we present the system developed by IHS-RD-Belarus for automated measuring of semantic similarity between two sentences using support vector regression (SVR) implemented in LIBSVM toolbox1 (Chang and Lin, 2011) with multiple features representing similarity and dif- ference scores calculated for each pair of sentenc- es. The rest of the paper is structured as follows. In Section 2 we describe the task and the data used to train our system. Section 3 describes in detail the features used by our system. Results and a short conclusion are presented in Section 4 and 5, re- spectively. 2 Task description Semantic Textual Similarity (STS) measures the degree of equivalence in the underlying semantics of paired snippets of text. It can range from com- plete unrelatedness to exact semantic equivalence (Agirre et al., 2015). Given two sentences, participating systems are asked to return a continuous valued similarity score on a scale from 0 to 5, with 0 indicating that the semantics of the sentences are completely unre- lated and 5 signifying semantic equivalence. For example, the sentence “Military plane crashes in south France” and the sentence “Military plane crashes in southeastern Turkey, 1 dead” have a very low similarity score despite many equal words, thus scored 1.0, while the sentence “Sar- kozy announces re-election bid” and the sentence “France's Nicolas Sarkozy makes his reelection bid official” are scored 4.2, as they are considered very similar even though there are many word differ- ences between them. 1 https://www.csie.ntu.edu.tw/~cjlin/libsvm/ 696 2.1 Dataset The dataset used for training of our system consists of the datasets provided by the organizers of the STS shared task, specifically 1500 pairs of news- …

Content truncated to 500 words