RTM at SemEval-2017 Task 1: Referential Translation Machines for Predicting Semantic Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 203–207, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics RTM at SemEval-2017 Task 1: Referential Translation Machines for Predicting Semantic Similarity Ergun Bic¸ici orcid.org/0000-0002-2293-2031 bicici.github.com Abstract We use referential translation machines for predicting the semantic similarity of text in all STS tasks which contain Arabic, English, Spanish, and Turkish this year. RTMs pioneer a language independent ap- proach to semantic similarity and remove the need to access any task or domain spe- cific information or resource. RTMs be- come 6th out of 52 submissions in Span- ish to English STS. We average prediction scores using weights based on the training performance to improve the overall perfor- mance. 1 Referential Translation Machines (RTMs) Semantic textual similarity (STS) task (Cer et al., 2017) at SemEval-2017 (Bethard et al., 2017) is about quantifying the degree of similarity between two given sentences S1 and S2 in the same lan- guage or in different languages. RTMs use inter- pretants, data close to the task instances, to derive features measuring the closeness of the test sen- tences to the training data, the difficulty of trans- lating them, and to identify translation acts be- tween any two data sets for building prediction models. RTMs are applicable in different domains and tasks and in both monolingual and bilingual settings. Figure 1 depicts RTMs and explains the model building process. RTMs use ParFDA (Bic¸ici, 2016a) for in- stance selection and machine translation perfor- mance prediction system (MTPPS) (Bic¸ici and Way, 2015) for generating features for the training and the test set mapping both to the same space where the total number of features in each task becomes 368. The new features we include are about punctuation: number of tokens about punc- Figure 1: RTM depiction: ParFDA selects inter- pretants close to the training and test data using parallel corpus in bilingual settings and mono- lingual corpus in the target language or just the monolingual target corpus in monolingual set- tings; an MTPPS use interpretants and training data to generate training features and another use interpretants and test data to generate test features in the same feature space; learning and prediction takes place taking these features as input. tuation (Kozlova et al., 2016) and the cosine be- tween the punctuation vectors. RTMs are providing a language independent text processing and machine learning model able to use predictions from different predictors. We use ridge regression (RR), k-nearest neighors (KNN), support vector regression (SVR), Ad- aBoost (Freund and Schapire, 1997), and ex- tremely randomized trees (TREE) (Geurts et al., 2006) as learning models in combination with fea- ture selection (FS) (Guyon et al., 2002) and partial least squares (PLS) (Wold et al., 1984). For most of the models, we use scikit-learn. 1 We optimize the models using a subset of the train- ing data for the following parameters: λ for RR, k for KNN, γ, C, and ϵ for SVR, minimum number of samples for leaf nodes …

Content truncated to 500 words