HCCL at SemEval-2017 Task 2: Combining Multilingual Word Embeddings and Transliteration Model for Semantic Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 220–225, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics HCCL at SemEval-2017 Task 2: Combining Multilingual Word Embeddings and Transliteration Model for Semantic Similarity Junqing He1,3, Long Wu1,3, Xuemin Zhao1, Yonghong Yan1,2,3 1The Key Laboratory of Speech Acoustics and Content Understanding Institute of Acoustics, Chinese Academy of Sciences 2Xinjiang Laboratory of Minority Speech and Language Information Processing Xinjiang Technical Institute of Physics and Chemistry, Chinese Academy of Sciences 3University of Chinese Academy of Sciences {hejunqing,wulong,zhaoxuemin,yonghongyan}@hccl.ioa.ac.cn Abstract In this paper, we introduce an approach to combining word embeddings and ma- chine translation for multilingual seman- tic word similarity, the task2 of SemEval- 2017. Thanks to the unsupervised translit- eration model, our cross-lingual word em- beddings encounter decreased sums of OOVs. Our results are produced using only monolingual Wikipedia corpora and a limited amount of sentence-aligned data. Although relatively little resources are uti- lized, our system ranked 3rd in the mono- lingual subtask and can be the 6th in the cross-lingual subtask. 1 Introduction With convenient word representation methods be- ing proposed, word embeddings are successfully utilized in state-of-the-art systems ranging from text classification (Kim, 2014), opinion catego- rization (Enr´ıquez et al., 2016), machine transla- tion (Zou et al., 2013), to stock price prediction (Peng and Jiang, 2016) and so on. In earlier studies, the latent semantic analysis (LSA) was introduced by Deerwester (1990). It is called topic model because terms are represented as the vectors of topics and was popularized by Landauer (1997). In 2003, researchers developed the topic model based on latent Dirichlet alloca- tion(LDA) (Blei et al., 2003). LDA did not widely spread until the Gibbs sampling was applied to the on-line training of LDA (Hoffman et al., 2010). Another traditional distributional method, point- wise mutual information metric was proposed by Turney and Pental (2010). Recently, fast dis- tributed embeddings like (Mikolov et al., 2013c) and GloVe (Pennington et al., 2014) are based on the assumption that the meaning of a word de- pends on its context. As Levy et al. (2015) pointed out, there is no significant performance difference between them. For cross-lingual word representation, there are generally four categories: Monolingual map- ping (Mikolov et al., 2013b), pseudo-cross-lingual training (Gouws and Søgaard, 2015), cross-lingual training (Hermann and Blunsom, 2014) and joint optimization (Coulmance et al., 2015). As pre- sented in (Mogadala and Rettinger, 2016) , the joint optimization method represents the state-of- the-art level in cross-lingual text classification and translation. These methods train embeddings both on monolingual and parallel corpora by jointly op- timizing the losses. However, they are rarely used in word similarity due to the unsatisfying perfor- mance. In this task, we adopt different strategies for the two subtasks. We use word2vec for subtask1, monolingual word similarity. For the subtask2, cross-lingual word similarity, we use jointly op- timized cross-lingual word representation in ad- dition to transliteration model. We build a cross- lingual word embedding system and a special ma- chine translation …

Content truncated to 500 words