ResSim at SemEval-2017 Task 1: Multilingual Word Representations for Semantic Textual Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 154–158, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics ResSim at SemEval-2017 Task 1: Multilingual Word Representations for Semantic Textual Similarity Johannes Bjerva Center for Language and Cognition Groningen University of Groningen The Netherlands j.bjerva@rug.nl Robert ¨Ostling Department of Linguistics Stockholm University Sweden robert@ling.su.se Abstract Shared Task 1 at SemEval-2017 deals with assessing the semantic similarity between sentences, either in the same or in differ- ent languages. In our system submission, we employ multilingual word representa- tions, in which similar words in different languages are close to one another. Us- ing such representations is advantageous, since the increasing amount of available parallel data allows for the application of such methods to many of the languages in the world. Hence, semantic similarity can be inferred even for languages for which no annotated data exists. Our system is trained and evaluated on all language pairs included in the shared task (English, Span- ish, Arabic, and Turkish). Although de- velopment results are promising, our sys- tem does not yield high performance on the shared task test sets. 1 Introduction Semantic Textual Similarity (STS) is the task of assessing the degree to which two sentences are semantically similar. Within the SemEval STS shared tasks, this is measured on a scale ranging from 0 (no semantic similarity) to 5 (complete se- mantic similarity) (Cer et al., 2017). Monolingual STS is an important task, for instance for evalua- tion of machine translation (MT) systems, where estimating the semantic similarity between a sys- tem’s translation and the gold translation can aid both system evaluation and development. The task is already a challenging one in a monolingual set- ting, e.g., when estimating the similarity between two English sentences. In this paper, we tackle the more difficult case of cross-lingual STS, e.g., es- timating the similarity between an English and an Arabic sentence. Previous approaches to this problem have fo- cussed on two main approaches. On the one hand, MT approaches have been attempted (e.g. Lo et al. (2016)), which allow for monolingual similarity assessment, but suffer from the fact that involv- ing a fully-fledged MT system severely increases system complexity. Applying bilingual word rep- resentations, on the other hand, bypasses this issue without inducing such complexity (e.g. Aldarmaki and Diab (2016)). However, bilingual approaches do not allow for taking advantage of the increas- ing amount of STS data available for more than one language pair. Currently, there are several methods available for obtaining high quality multilingual word rep- resentations. It is therefore interesting to investi- gate whether language can be ignored entirely in an STS system after mapping words to their re- spective representations. We investigate the utility of multilingual word representations in a cross- lingual STS setting. We approach this by com- bining multilingual word representations with a deep neural network, in which all parameters are shared, regardless of language combinations. The contributions of this paper can be summed as follows: i) …

Content truncated to 500 words