ECNU at SemEval-2016 Task 1: Leveraging Word Embedding From Macro and Micro Views to Boost Performance for Semantic Textual Similarity

Proceedings of SemEval-2016, pages 621–627, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics ECNU at SemEval-2016 Task 1: Leveraging Word Embedding from Macro and Micro Views to Boost Performance for Semantic Textual Similarity Junfeng Tian1, Man Lan1,2∗ 1Department of Computer Science and Technology, East China Normal University, Shanghai, P.R.China 2Shanghai Key Laboratory of Multidimensional Information Processing 51151201048@ecnu.cn, mlan@cs.ecnu.edu.cn∗ Abstract This paper presents our submissions for se- mantic textual similarity task in SemEval 2016. Based on several traditional fea- tures (i.e., string-based, corpus-based, ma- chine translation similarity and alignmen- t metrics), we leverage word embedding from macro (i.e., first get representation of sen- tence, then measure the similarity of sentence pair) and micro views (i.e., measure the simi- larity of word pairs separately) to boost perfor- mance. Due to the various domains of train- ing data and test data, we adopt three differ- ent strategies: 1) U-SEVEN: an unsupervised model, which utilizes seven straight-forward metrics; 2) S1-All: using all available dataset- s; 3) S2: selecting the most similar training sets for each test set. Results on test sets show that the unified supervised model (i.e., S1-All) achieves the best averaged performance with a mean correlation of 75.07%. 1 Introduction Estimating the degree of semantic similarity be- tween two sentences is the building block of many Natural Language Processing (NLP) applications, such as question answering, textual entailment, tex- t summarization etc. Therefore, Semantic Textual Similarity (STS) has received an increasing amoun- t of attention in recent years, e.g., the STS tasks in Semantic Evaluation Exercises have been held from 2012 to 2016. To identify semantic similarity of sentence pairs, most existing works adopt at least one of the follow- ing feature types: 1) string based similarity (B¨ar et al., 2012; Jimenez et al., 2012) which employs com- mon functions to calculate similarities over string sequences extracted from original strings, e.g., lem- ma, stem, or n-grams sequences; 2) corpus based similarity (ˇSari´c et al., 2012) where distributional models such as Latent Semantic Analysis (LSA), are used to derive the distributional vectors of word- s from a large corpus according to their occurrence patterns, afterwards, similarities of sentence pairs are calculated using these vectors; 3) knowledge based method (Agirre et al., 2015b) which estimates the similarities with the aid of external resources, such as WordNet. Among them, Sultan et al. (2015) leverage different word alignment strategies to bring word-level similarity to sentence-level similarity. Traditional NLP feature engineering often treat sentence as a bag of words or term frequency, and endeavor to evaluate the similarity according to the co-occurrence of words or other replacement word- s. For example, Zhao et al. (2014) built a super- vised model using ensemble of heterogeneous fea- tures and achieved great performance on STS Task 2014. However, it is difficult to evaluate semantic re- latedness if all the word in both sentences is unique. For example: A storm will spread snow over Shang- hai; The earthquakes have shaken parts of Okla- homa. These sentences have no words …

Content truncated to 500 words