JUNITMZ at SemEval-2016 Task 1: Identifying Semantic Similarity Using Levenshtein Ratio

Proceedings of SemEval-2016, pages 702–705, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics JUNITMZ at SemEval-2016 Task 1: Identifying Semantic Similarity Using Levenshtein Ratio Sandip Sarkar Computer Science and Engineering Jadavpur University, Kolkata sandipsarkar.ju@gmail.com Dipankar Das Computer Science and Engineering Jadavpur University, Kolkata dipankar.dipnil2005@gmail.com Partha Pakray Computer Science and Engineering NIT Mizoram, Mizoram parthapakray@gmail.com Alexander Gelbukh Center for Computing Research Instituto Politcnico Nacional, Mexico gelbukh@gelbukh.com Abstract In this paper we describe the JUNITMZ 1 sys- tem that was developed for participation in Se- mEval 2016 Task 1: Semantic Textual Simi- larity. Methods for measuring the textual sim- ilarity are useful to a broad range of applica- tions including: text mining, information re- trieval, dialogue systems, machine translation and text summarization. However, many sys- tems developed specifically for STS are com- plex, making them hard to incorporate as a module within a larger applied system. In this paper, we present an STS system based on three simple and robust similarity features that can be easily incorporated into more complex applied systems. The shared task results show that on most of the shared tasks evaluation sets, these signals achieve a strong (>0.70) level of correlation with hu- man judgements. Our system’s three features are: unigram overlap count, length normalized edit distance and the score computed by the METEOR machine translation metric. Fea- tures are combined to produces a similarity prediction using both a feedforward and recur- rent neural network. 1 Introduction Semantic similarity plays important role in many natural language processing (NLP) applications. The semantic textual similarity (STS) shared task has been held annually since 2012 in order to assess different approaches to computing textual similarity 1This work was supported by the Project No. YSS/2015/000988 of National Institute of Technology Mizoram. across a variety of different domains. Research systems developed specifically for the STS task have resulted in a progression of systems that achieve increasing levels of performance but that are often also increasingly more complex. Complex ap- proaches may be difficult if not impossible to incor- porate as a component of larger applied NLP sys- tems. The system described in this paper explores an al- ternative approach based on three simple and ro- bust textual similarity features. Our features are simple enough that they can be easily incorporated into larger applied systems that could benefit from textual similarity scores. The first feature simply counts the number of words common to the pair of sentences being assessed. The second provides the length normalized edit distance to transform one sentence into another. The final feature scores the two sentences using the METEOR machine transla- tion metric. The latter allows the reuse of the lin- guistic analysis modules developed within the ma- chine translation community to assess translation quality. METEOR’s implementation of these mod- ules is lightweight and efficient, making it not overly cumbersome to incorporate features based on ME- TEOR into larger applied systems. The remainder of this paper is structured as follows. Section 2 provides an overview of our …

Content truncated to 500 words