MiniExperts: An SVM Approach for Measuring Semantic Textual Similarity

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 96–101, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics MiniExperts: An SVM Approach for Measuring Semantic Textual Similarity Hanna B´echara∗a, Hernani Costa∗b, Shiva Taslimipoora, Rohit Guptaa, Constantin Or˘asana, Gloria Corpas Pastorb and Ruslan Mitkova aRIILP, University of Wolverhampton, UK bLEXYTRAD, University of Malaga, Spain {hanna.bechara,hercos,shiva.taslimi,r.gupta, c.orasan,gcorpas,r.mitkov}@{awlv.ac.uk,buma.es} ∗These two authors contributed equally to this work. Abstract This paper describes the system submitted by the University of Wolverhampton and the University of Malaga for SemEval-2015 Task 2: Semantic Textual Similarity. The system uses a Supported Vector Machine approach based on a number of linguistically motivated features. Our system performed satisfactorily for English and obtained a mean 0.7216 Pearson correlation. However, it performed less adequately for Spanish, obtaining only a mean 0.5158. 1 Introduction Similarity measures play an important role in a wide variety of Natural Language Processing (NLP) applications. Information Retrieval (IR), for example, relies on semantic similarity in order to determine the best result for a related query. Semantic similarity also plays a crucial role in other applications such as Paraphrasing and Translation Memory (TM). However, computing semantic similarity between sentences remains a complex and difficult task. Over the years, SemEval’s shared tasks worked to fine-tune and perfect these similarity measures, and explore the nature of meaning in language. SemEval2015’s Task 2 involves computing how similar two sentences are in both English (Subtask 2a) and Spanish (Subtask 2b). In this paper we detail our submission to SemEval Task 2. We use an improved and revised version of the system presented in our SemEval 2014 submission (Gupta et al., 2014). As in Gupta et al., 2014, we employ a Machine Learning (ML) method which exploits available NLP technology, adding features inspired by deep semantics (such as parsing and paraphrasing) with distributional Similarity Measures, Conceptual Similarity Measures, Semantic Similarity Measures and Corpus Pattern Analysis1 (CPA). The remainder of the paper is structured as follows. Section 2 describes our approach, i.e. explains how the data was preprocessed and what features were extracted. Section 3 is divided in two section, the first one describes the ML algorithm and how it was tuned for this task (section 3.1) and the second one shows the obtained results along with a descriptive analysis of the runs based on the test and training data provided by the SemEval-2015 Task 2 (section 3.2). Finally, section 4 presents the final remarks and highlights our future plans for improving the system. 2 Approach This section describes our approach to calculating semantic relatedness. It covers all the required preprocessing steps to extract the features themselves. 2.1 Data Preprocessing This section presents all the tools, libraries and frameworks used to preprocess not only the test datasets but also the training datasets. 2.1.1 POS-Tagger, Lemmatiser, Stemmer The software we used for these specific NLP tasks were: the Stanford CoreNLP2 (Toutanova et al., 1http://pdev.org.uk 2http://nlp.stanford.edu/software/ corenlp.shtml 96 2003) toolkit, which provides a lemmatiser, POS- Tagger, NER, parsing, and coreference; the TT4J3 library, …

Content truncated to 500 words