ECNU at SemEval-2017 Task 1: Leverage Kernel-based Traditional NLP features and Neural Networks to Build a Universal Model for Multilingual and Cross-lingual Semantic Textual Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 191–197, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics ECNU at SemEval-2017 Task 1: Leverage Kernel-based Traditional NLP features and Neural Networks to Build a Universal Model for Multilingual and Cross-lingual Semantic Textual Similarity Junfeng Tian1, Zhiheng Zhou1, Man Lan1,2∗, Yuanbin Wu1,2 1Department of Computer Science and Technology, East China Normal University, Shanghai, P.R.China 2Shanghai Key Laboratory of Multidimensional Information Processing {jftian, zhzhou}@stu.ecnu.edu.cn {mlan, ybwu}@cs.ecnu.edu.cn Abstract To model semantic similarity for multilin- gual and cross-lingual sentence pairs, we first translate foreign languages into En- glish, and then build an efficient mono- lingual English system with multiple NLP features. Our system is further support- ed by deep learning models and our best run achieves the mean Pearson correlation 73.16% in primary track. 1 Introduction Sentence semantic similarity is the building block of natural language understanding. Previous Se- mantic Textual Similarity (STS) tasks in SemEval focused on judging sentence pairs in English and achieved great success. In SemEval-2017 STS shared task concentrates on the evaluation of sentence semantic similarity in multilingual and cross-lingual (Agirre et al., 2017). There are t- wo challenges in modeling multilingual and cross- lingual sentence similarity. On the one hand, this task requires human linguistic expertise to design specific features due to the different characteristic- s of languages. On the other hand, lack of enough training data for a particular language would lead to a poor performance. The SemEval-2017 STS shared task assesses the ability of participant systems to estimate the degree of semantic similarity between monolin- gual and cross-lingual sentences in Arabic, En- glish and Spanish, which is organized into a set of six secondary sub-tracks (Track 1 to Track 6) and a single combined primary track (Primary Track) achieved by submitting results for all of the sec- ondary sub-tracks. Specifically, track 1, 3 and 5 are to determine STS scores for monolingual sentence pairs in Arabic, Spain and English, re- spectively. Track 2, 4, and 6 involve estimat- ing STS scores for cross-lingual sentence pairs from the combination of two particular languages, i.e., Arabic-English, Spanish-English and surprise language (here is Turkish)-English cross-lingual pairs. Given two sentences, a continuous val- ued similarity score on a scale from 0 to 5 is re- turned, with 0 indicating that the semantics of the sentences are completely independent and 5 sig- nifying semantic equivalence. The system is as- sessed by computing the Pearson correlation be- tween system returned semantic similarity scores and human judgements. To address this task, we first translate all sen- tences into English through the state-of-the-art machine translation (MT) system, i.e., Google Translator1. Then we adopt a combination method to build a universal model to estimate seman- tic similarity, which consists of traditional natu- ral language processing (NLP) methods and deep learning methods. For traditional NLP methods, we design multiple effective NLP features to de- pict the semantic matching degree and then su- pervised machine learning-based regressors are trained to make prediction. …

Content truncated to 500 words