ECNU: Using Traditional Similarity Measurements and Word Embedding for Semantic Textual Similarity Estimation

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 117–122, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics ECNU: Using Traditional Similarity Measurements and Word Embedding for Semantic Textual Similarity Estimation Jiang Zhao, Man Lan∗ , Jun Feng Tian Shanghai Key Laboratory of Multidimensional Information Processing Department of Computer Science and Technology, East China Normal University, Shanghai 200241, P. R. China 51121201042,10112130275@ecnu.cn; mlan@cs.ecnu.edu.cn∗ Abstract This paper reports our submissions to seman- tic textual similarity task, i.e., task 2 in Se- mantic Evaluation 2015. We built our sys- tems using various traditional features, such as string-based, corpus-based and syntactic simi- larity metrics, as well as novel similarity mea- sures based on distributed word representa- tions, which were trained using deep learning paradigms. Since the training and test datasets consist of instances collected from various do- mains, three different strategies of the usage of training datasets were explored: (1) use all available training datasets and build a unified supervised model for all test datasets; (2) se- lect the most similar training dataset and sep- arately construct a individual model for each test set; (3) adopt multi-task learning frame- work to make full use of available training set- s. Results on the test datasets show that using all datasets as training set achieves the best av- eraged performance and our best system ranks 15 out of 73. 1 Introduction Estimating the degree of semantic similarity be- tween two sentences is the building block of many natural language processing (NLP) applications, such as textual entailment (Zhao et al., 2014a), tex- t summarization (Lloret et al., 2008), question an- swering (Celikyilmaz et al., 2010), etc. Therefore, semantic textual similarity (STS) has been received an increasing amount of attention in recent years, e.g., the Semantic Textual Similarity competition- s in Semantic Evaluation Exercises have been held from 2012 to 2014. This year the participants in the STS task in SemEval 2015 (Agirre et al., 2015) are required to rate the similar degree of a pair of sen- tences by a value from 0 (no relation) to 5 (semantic equivalence) with an optional confidence score. To identify semantic textual similarity of tex- t pairs, most existing works adopt at least one of the following feature types: (1) string based simi- larity (B¨ar et al., 2012; Jimenez et al., 2012) which employs common functions to calculate similari- ties over string sequences extracted from original strings, e.g., lemma, stem, or n-gram sequences; (2) corpus based similarity (ˇSari´c et al., 2012; Han et al., 2013) where distributional models such as Laten- t Semantic Analysis (LSA) (Landauer and Dumais, 1997), are used to derive the distributional vectors of words from a large corpus according to their occur- rence patterns, afterwards, similarities of sentence pairs are calculated using these vectors; (3) knowl- edge based method (Shareghi and Bergler, 2013; Mihalcea et al., 2006) which estimates the similari- ties with the aid of external resources, such as Word- Net1. Among them, lots of researchers (Sultan et al., 2014; …

Content truncated to 500 words