NORMAS at SemEval-2016 Task 1: SEMSIM: A Multi-Feature Approach to Semantic Text Similarity

Proceedings of SemEval-2016, pages 718–725, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics NORMAS at SemEval-2016 Task 1: SEMSIM: A Multi-Feature Approach to Semantic Text Similarity Adebayo Kolawole John and Luigi Di Caro and Guido Boella University of Torino Corso Svizzera 185 Torino, 10149, Italy kolawolejohn.adebayo@unibo.it, (dicaro, guido)@di.unito.it Abstract This paper presents the submission of our team (NORMAS) to the SemEval 2016 se- mantic textual similarity (STS) shared task. We submitted three system runs, each using a set of 36 features extracted from the training set. The runs explore the use of the following three machine learning algorithms: Support Vector Regression, Elastic Net and Random Forest. Each run was trained using sentence pairs from the STS 2012 training data. Fea- tures extracted include lexical, syntactic and semantic features. This paper describes the features we designed for assessing the seman- tic similarity between sentence pairs, the mod- els we build using these features and the per- formance obtained by the resulting systems on the 2016 evaluation data. 1 Introduction Computationally assessing the semantic similarity of natural language data has gained the attention of researchers in the field of computational linguistics. Machines that are able to quantify the semantics of natural language have numerous important applica- tions including: Document Similarity (Elsayed et al., 2008; Huang, 2008), Word Similarity (Resnik, 1995; Dagan, 2000; Bollegala et al., 2007; Pedersen et al., 2004), Text Summarization (Barzilay and El- hadad, 1999; Gong and Liu, 2001), Informtion Re- trieval (Salton and Buckley, 1988; Manning et al., 2008), Plagiarism detection (Si et al., 1997), Para- phrase detection (Fernando and Stevenson, 2008) and especially Machine Translation (Brown et al., 1990). The semantic textual similarity (STS) (Agirre et al., 2012) task measures the level of semantic equiv- alence between two approximately sentence sized snippets of texts on a graded scale from 0 (unrelated) to 5 (completely equivalent). This paper describes our participation in the SemEval 2016 STS shared task (Agirre et al., 2016). Our system explores 36 lexical and semantic features (e.g., string matching, WordNet similarity, word overlap) in combination with three very distinct learning algorithms (Support Vector Regression, Elastic Net and Random forest). The remainder of this paper is organized as fol- lows: Section 2 describes our feature set in detail and our approach to feature selection. Section 3 de- scribes our machine learning models with section 4 presenting our results on the shared task evaluation data. 2 Feature Generation We used a mixture of lexical, syntactic and seman- tic features extracted from text. Below, we describe each of the features provided to our model. 2.1 Lexical Features String matching: String matching techniques com- pare words in two texts character by character and can be used to approximately capture morphological differences in the terms. B¨ar et al. (2012), one of the best performing systems in 2012, used variants of string matching algorithms. Our system made use of features computed using the following string match- ing methods: 1. Longest common Substring: This obtains the longest sequences of …

Content truncated to 500 words