Extending Monolingual Semantic Textual Similarity Task to Multiple Cross-lingual Settings

Extending Monolingual Semantic Textual Similarity Task to Multiple Cross-lingual Settings Yoshihiko Hayashi† and Wentao Luo‡ †Faculty of Science and Engineering, Waseda University 2-4-12 Ohukubo, Shinjuku, Tokyo, 169-0072 Japan yshk.hayashi@aoni.waseda.jp ‡Graduate School of Language and Culture, Osaka University 1-8 Machikaneyama, Toyonaka, Osaka, 560-0043 Japan u172761i@ecs.osaka-u.ac.jp Abstract This paper describes our independent effort for extending the monolingual semantic textual similarity (STS) task setting to multiple cross-lingual settings involving English, Japanese, and Chinese. So far, we have adopted a “monolingual similarity after translation” strategy to predict the semantic similarity between a pair of sentences in different languages. With this strategy, a monolingual similarity method is applied after having (one of) the target sentences translated into a pivot language. Therefore, this paper specifically details the required and developed resources to implement this framework, while presenting our current results for English-Japanese-Chinese cross-lingual STS tasks that may exemplify the validity of the framework. Keywords: semantic textual similarity, STS, cross-lingual semantic similarity, machine translation quality 1. Introduction A Semantic Textual Similarity (STS) measures the de- gree of semantic equivalence between two target sen- tences (Agirre et al., 2012; Agirre et al., 2013; Agirre et al., 2014). Given its potential in several types of NLP ap- plications, the series of STS shared tasks1 accumulates a range of approaches for predicting semantic textual simi- larity. However the previous STS tasks have been focusing on only monolingual task in English or Spanish: Just only recently, a cross-lingual task (SemEval 2016 Task 12) has been proposed, but the target languages are still limited to English and Spanish. This paper describes our independent effort for extending the monolingual STS task setting to multiple cross-lingual settings involving English, Japanese, and Chinese. Our cur- rent framework (section 2) adopts a “monolingual similar- ity after translation” strategy: We first convert the given target sentence pair in different languages into a pivot lan- guage, and then apply a method to predict the monolingual similarity between in the pivot language. Note however that the pivot language can be one of the languages of target sen- tence pair, or it could be a third language. Resources (section 3) developed for enabling this frame- work, as well as the current results (section 4) are shown to exemplify the fundamental validity of our framework. Fu- ture directions are discussed while summarizing the current achievements and the issues (section 5). 2. Proposed framework Figure 1 overviews our current framework for predicting cross-lingual semantic textual similarities. It implements “monolingual similarity after translation” strategy, in which a pair of target sentences (S1, S2) in different languages 1http://ixa2.si.ehu.es/stswiki/index.php/ Main_Page 2http://alt.qcri.org/semeval2016/task1/ (L1 and L2, respectively) are first translated into a pivot language (PL), and then a method for predicting monolin- gual textual similarity is applied. Note however that the PL can be either of L1 or L2; in that case, translation of the sentence in the PL is obviously not necessary. We have evaluated three approaches for predicting mono- lingual semantic textual similarity, which are referred to as ML, AL, and ML+ respectively. • The …

Content truncated to 500 words