Lump at SemEval-2017 Task 1: Towards an Interlingua Semantic Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 144–149, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics Lump at SemEval-2017 Task 1: Towards an Interlingua Semantic Similarity Cristina Espa˜na-Bonet 1University of Saarland 2DFKI, German Research Center for Artificial Intelligence Saarbr¨ucken, Germany cristinae@dfki.de Alberto Barr´on-Cede˜no Qatar Computing Research Institute HBKU, Qatar albarron@hbku.edu.qa albarron@gmail.com Abstract This is the Lump team participation at Se- mEval 2017 Task 1 on Semantic Textual Similarity. Our supervised model relies on features which are multilingual or interlin- gual in nature. We include lexical sim- ilarities, cross-language explicit semantic analysis, internal representations of mul- tilingual neural networks and interlingual word embeddings. Our representations al- low to use large datasets in language pairs with many instances to better classify in- stances in smaller language pairs avoid- ing the necessity of translating into a sin- gle language. Hence we can deal with all the languages in the task: Arabic, English, Spanish, and Turkish. 1 Introduction The Semantic Textual Similarity (STS) task poses the following challenge. Let s and t be two text snippets. Determine the degree of equivalence α(s, t) | α ∈ [0, 5]. Whereas 0 represents com- plete independence, 5 reflects semantic equiva- lence. The current edition (Cer et al., 2017) in- cludes the monolingual ar–ar, en–en, and es– es, as well as the cross-language ar–en, es– en, and tr–enlanguage pairs. We use the two- letter ISO 639-1 codes: ar=Arabic, en=English, es=Spanish, and tr=Turkish. Multilinguality is the premise of the Lump ap- proach: we use representations which lie towards language-independence as we aim to be able to approach similar tasks on other languages, pay- ing the least possible effort. Our regression model relies on different kinds of features, from simple length-based and lexical similarities to more so- phisticated embeddings and deep neural net inter- nal representations. 2 Features Description The main algorithm used in this work is the sup- port vector regressor from LibSVM (Chang and Lin, 2011). We use an RBF kernel and greed- ily select the best parameters by 5-fold cross- validation. In addition, we experiment with a dif- ferent machine learning component built with gra- dient boosting algorithms as implemented by the XGBoost package.1 We describe the features in growing level of complexity: from language flags up to embed- dings derived from neural machine translation. 2.1 Language-Identification Flags (6 feats.) The novelty of the cross-language tasks causes a noticeable language imbalance in the amount of data (cf. Table 1). To deal with this issue, one of our systems learns on the instances in all the lan- guage pairs jointly. In order to reduce the clutter of the different data distributions, we devised six bi- nary features that mark the languages of each pair. lang1, lang2 and lang3 are set to 1 if s is written in either ar, en, or es, respectively. The other three features, lang4, lang5, and lang6, provide the same information for t. For instance, the value for the six features for a pair en–ar …

Content truncated to 500 words