PurdueNLP at SemEval-2017 Task 1: Predicting Semantic Textual Similarity with Paraphrase and Event Embeddings

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 198–202, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics PurdueNLP at SemEval-2017 Task 1: Predicting Semantic Textual Similarity with Paraphrase and Event Embeddings I-Ta Lee, Mahak Goindani, Chang Li, Di Jin, Kristen Marie Johnson Xiao Zhang, Maria Leonor Pacheco, Dan Goldwasser Department of Computer Science, Purdue University West Lafayette, IN 47907 {lee2226,mgoindan,li1873,jind,john1187, zhang923,pachecog,dgoldwas}@purdue.edu Abstract This paper describes our proposed solu- tion for SemEval 2017 Task 1: Semantic Textual Similarity (Daniel Cer and Spe- cia, 2017). The task aims at measuring the degree of equivalence between sentences given in English. Performance is eval- uated by computing Pearson Correlation scores between the predicted scores and human judgements. Our proposed system consists of two subsystems and one re- gression model for predicting STS scores. The two subsystems are designed to learn Paraphrase and Event Embeddings that can take the consideration of paraphras- ing characteristics and sentence structures into our system. The regression model associates these embeddings to make the final predictions. The experimental re- sult shows that our system acquires 0.8 of Pearson Correlation Scores in this task. 1 Introduction The SemEval Semantic Textual Similarity (STS) task (Daniel Cer and Specia, 2017) is to assess the degree of similarity between two given sen- tences and assign a score on a scale from 0 to 5. A score of 0 indicates that the two sentences are completely dissimilar, while a score of 5 indicates that the sentences have the same meaning. Predict- ing the similarity between pieces of texts finds util- ity in many NLP tasks such as question-answering, and plagiarism detection. In this paper, we proposed a system to fa- cilitate STS task. Our system includes training two types of embeddings–Paraphrase Embeddings (PE) and Event Embeddings (EE)–as features to assess STS. For the first type of embeddings, PE, we exploit two crucial properties for measuring sentence similarity: paraphrasing characteristics and sentence structures. The paraphrasing char- acteristics help identifying if two sentences share the same meaning. Our system incorporates it us- ing an unsupervised learning step over the Para- phrase Database (PPDB; Ganitkevitch et al. 2013), which is inspired by Wieting et al. 2015a. The sen- tence structure, on the other hand, can detect struc- tural differences, which reflect different aspects of the similarity between the input sentences. Our system employs a Convolutional Neural Network (CNN) to strengthen the embedding by including the sentence structure into our representation. The second type of embeddings, EE, conveys the dis- tributional semantics of events in a narrative set- ting, associating a vector with each event. In the last part of our system, we build a regres- sion model that associates the two distributed rep- resentations and predicts the similarity scores. 2 System Description Our system builds two types of embedding mod- els, Paraphrase Embeddings (PE) and Event Em- beddings (EE), and trains a regression model for predicting the similarity score between two sen- tences, which is described in this Section 2.3. 2.1 …

Content truncated to 500 words