HLTC-HKUST: A Neural Network Paraphrase Classifier using Translation Metrics, Semantic Roles and Lexical Similarity Features

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 23–28, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics HLTC-HKUST: A Neural Network Paraphrase Classifier using Translation Metrics, Semantic Roles and Lexical Similarity Features Dario Bertero, Pascale Fung Human Language Technology Center Department of Electronic and Computer Engineering The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong dbertero@ust.hk, pascale@ece.ust.hk Abstract This paper describes the system developed by our team (HLTC-HKUST) for task 1 of Se- mEval 2015 workshop about paraphrase clas- sification and semantic similarity in Twitter. We trained a neural network classifier over a range of features that includes translation met- rics, lexical and syntactic similarity score and semantic features based on semantic roles. The neural network was trained taking into consid- eration in the objective function the six dif- ferent similarity levels provided in the corpus, in order to give as output a more fine-grained estimation of the similarity level of the two sentences, as required by subtask 2. With an F-score of 0.651 in the binary paraphrase clas- sification subtask 1, and a Pearson coefficient of 0.697 for the sentence similarity subtask 2, we achieved respectively the 6th place and the 3rd place, above the average of what obtained by the other contestants. 1 Introduction Paraphrase identification is the problem to determine whether two sentences have the same meaning, and is the objective of the task 1 of SemEval 2015 workshop (Xu et al., 2015). Conventionally this task has been mainly evaluated on the Microsoft Research Paraphrase corpus (Dolan and Brockett, 2005), which consists of pairs of sen- tences taken out from news headlines and articles. News domain sentences are usually grammatically correct and of average to long length. The current state-of-the-art method to our knowledge on this cor- pus (Ji and Eisenstein, 2013) trains an SVM over latent semantic vectors, lexical and syntactic simi- larity features. Although their main objective was to show the effectiveness of a method based on la- tent semantic analysis, it is also evident that other features pertinent to different aspects of sentence sim- ilarity are able to boost the results. Previously Socher et al. (2011) used a recursive autoencoder to simi- larly obtain a vector representation of each sentence, again combining other lexical similarity features to improve the results. Other methods, such as Mad- nani et al. (2012) or Wan et al. (2006) used instead a more traditional supervised classification approach over different sets of features and different classifiers, most of which improved previous results. Task 1 of SemEval 2015 workshop required to evaluate paraphrases on a new corpus, consisting of sentences taken from Twitter posts (Xu et al., 2014). Twitter sentences notoriously differ from those taken from news articles: the 140 characters limit makes the sentences short, with few words, lots of different abbreviations; they also include many misspelled and invented words, and often lack a correct grammatical structure. Another important difference is the six- level classification labels provided, compared to the …

Content truncated to 500 words