Examining the Tip of the Iceberg: A Data Set for Idiom Translation

Examining the Tip of the Iceberg: A Data Set for Idiom Translation Marzieh Fadaee1, Arianna Bisazza2, Christof Monz1 1Informatics Institute, University of Amsterdam, Science Park 904, 1098 XH Amsterdam, The Netherlands 2Leiden Institute of Advanced Computer Science, Leiden University, 2333 CA Leiden, The Netherlands {m.fadaee,c.monz}@uva.nl a.bisazza@liacs.leidenuniv.nl Abstract Neural Machine Translation (NMT) has been widely used in recent years with significant improvements for many language pairs. Although state-of-the-art NMT systems are generating progressively better translations, idiom translation remains one of the open challenges in this field. Idioms, a category of multiword expressions, are an interesting language phenomenon where the overall meaning of the expression cannot be composed from the meanings of its parts. A first important challenge is the lack of dedicated data sets for learning and evaluating idiom translation. In this paper we address this problem by creating the first large-scale data set for idiom translation. Our data set is automatically extracted from a widely used German↔English translation corpus and includes, for each language direction, a targeted evaluation set where all sentences contain idioms and a regular training corpus where sentences including idioms are marked. We release this data set and use it to perform preliminary NMT experiments as the first step towards better idiom translation. Keywords: multiword expression, idioms, bilingual corpora, machine translation 1. Introduction Neural Machine Translation (NMT) (Bahdanau et al., 2015; Sutskever et al., 2014; Cho et al., 2014) has achieved sub- stantial improvements in translation quality over traditional Rule-based and Phrase-based Translation (PBMT) in re- cent years. For instance, subject-verb agreement, double- object verbs, and overlapping subcategorization are various areas where NMT successfully overcomes the limitations of PBMT (Isabelle et al., 2017; Bentivogli et al., 2016). How- ever, one of the remaining challenges of NMT is translating infrequent words and phrases (Koehn and Knowles, 2017; Fadaee et al., 2017) and idioms are a particular instance of this problem (Isabelle et al., 2017). Idioms are semantic lexical units whose meaning is often not simply a function of the meaning of its constituent parts (Nunberg et al., 1994; K¨ovecses and Szabo, 1996). The non-compositionality characteristic of idiom expres- sions exists in different degrees in a language (Nunberg et al., 1994). In English for example, for the idiom “spill the beans”, the word ‘spill’ symbolizes ‘reveal’ and ‘beans’ symbolizes the ‘secrets’. With the idiomatic expression “kick the bucket”, on the other hand, no such analysis is possible. Isabelle et al. (2017) builds a challenge set of 108 short sentences that each focus on one particular difficult phe- nomenon of the language. Their manual assessment of the eight sentences consisting of an idiomatic phrase show that NMT systems struggle with the translation of these phrases. The challenge of translating idiom phrases in NMT is partly due to the underlying complexity of identifying a phrase as idiomatic and generating its correct non-literal translation, and partly to the fact that idioms are rarely encountered in the standard data sets used for training NMT systems. As an example, in Table 1 we provide …

Content truncated to 500 words