UoW: NLP techniques developed at the University of Wolverhampton for Semantic Similarity and Textual Entailment

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 785–789, Dublin, Ireland, August 23-24, 2014. UoW: NLP Techniques Developed at the University of Wolverhampton for Semantic Similarity and Textual Entailment Rohit Gupta, Hanna B´echara, Ismail El Maarouf and Constantin Or˘asan Research Group in Computational Linguistics, Research Institute of Information and Language Processing, University of Wolverhampton, UK {R.Gupta, Hanna.Bechara, I.El-Maarouf, C.Orasan}@wlv.ac.uk Abstract This paper presents the system submit- ted by University of Wolverhampton for SemEval-2014 task 1. We proposed a ma- chine learning approach which is based on features extracted using Typed Depen- dencies, Paraphrasing, Machine Transla- tion evaluation metrics, Quality Estima- tion metrics and Corpus Pattern Analysis. Our system performed satisfactorily and obtained 0.711 Pearson correlation for the semantic relatedness task and 78.52% ac- curacy for the textual entailment task. 1 Introduction The SemEval task 1 (Marelli et al., 2014a) in- volves two subtasks: predicting the degree of re- latedness between two sentences and detecting the entailment relation holding between them. The task uses SICK dataset (Marelli et al., 2014b), consisting of 10000 pairs, each annotated with re- latedness in meaning and entailment relationship holding between them. Similarity measures be- tween sentences are required in a wide variety of NLP applications. In applications like Informa- tion Retrieval (IR), measuring similarity is a vi- tal step in order to determine the best result for a related query. Other applications such as Para- phrasing and Translation Memory (TM) rely on similarity measures to weight results. However, computing semantic similarity between sentences is a complex and difficult task, due to the fact that the same meaning can be expressed in a variety of ways. For this reason it is necessary to have more than a surface-form comparison. We present a method based on machine learning which exploits available NLP technology. Our ap- This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ proach relies on features inspired by deep seman- tics (such as parsing and paraphrasing), machine translation quality estimation, machine translation evaluation and Corpus Pattern Analysis (CPA1). We use the same features to measure both se- mantic relatedness and textual entailment. Our hy- pothesis is that each feature covers a particular as- pect of implicit similarity and entailment informa- tion contained within the pair of sentences. Train- ing is performed in a regression framework for se- mantic relatedness and in a classification frame- work for textual entailment. The remainder of the paper is structured as fol- lows. In Section 2, we review the work related to our study and the existing NLP technologies used to measure sentence similarity. In Sections 3 and 4, we describe our approach and the similarity measures we used. In Section 5, we present the re- sults and an analysis of our runs based on the test and training data provided by the SemEval-2014 task. Finally, our work is summed up in Section 6 with perspectives for future …

Content truncated to 500 words