Better OOV Translation with Bilingual Terminology Mining

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5809–5815 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5809 Better OOV Translation with Bilingual Terminology Mining Matthias Huck, Viktor Hangya and Alexander Fraser Center for Information and Language Processing LMU Munich, Germany {mhuck, hangyav, fraser}@cis.lmu.de Abstract Unseen words, also called out-of-vocabulary words (OOVs), are difficult for machine trans- lation. In neural machine translation, byte-pair encoding can be used to represent OOVs, but they are still often incorrectly translated. We improve the translation of OOVs in NMT us- ing easy-to-obtain monolingual data. We look for OOVs in the text to be translated and trans- late them using simple-to-construct bilingual word embeddings (BWEs). In our MT exper- iments we take the 5−best candidates, which is motivated by intrinsic mining experiments. Using all five of the proposed target language words as queries we mine target-language sen- tences. We then back-translate, forcing the back-translation of each of the five proposed target-language OOV-translation-candidates to be the original source-language OOV. We show that by using this synthetic data to fine- tune our system the translation of OOVs can be dramatically improved. In our experiments we use a system trained on Europarl and mine sen- tences containing medical terms from mono- lingual data. 1 Introduction Neural machine translation (NMT) systems achieved a breakthrough in translation quality recently, by learning an end-to-end system (Sutskever et al., 2014; Bahdanau et al., 2015). However, NMT systems have low quality when translating out-of-vocabulary words (OOVs), especially because they have a fixed modest sized vocabulary due to memory limitations. By splitting words into subword units the problem of representing OOVs can be solved (Sennrich et al., 2016b) but their translation is still problematic because by definition source-side OOVs were not seen in the training parallel data together with their translations. In this work, we evaluate a simple approach for improving the translation of OOVs using bilingual word embeddings (BWEs), which we hope will trigger more research on this interesting problem. In previous approaches, to include words in the target sentence for which the translation is un- known the token unk is often used which can be handled by later steps. In many cases, such as named entities, it is possible to just copy the source token to the target side instead of translat- ing it. Gulcehre et al. (2016) proposed a pointer network based (Vinyals et al., 2015) system which can learn when to translate and when to copy. On the other hand, it is not possible to always copy when the translation is unknown. If the alignment of the unk tokens to the source are known it is possible to translate source words using a large dictionary as a post-processing step. Although NMT systems do not rely on word alignments ex- plicitly, it is possible to learn and output word alignments (Luong et al., 2015). It is also possi- ble to use lexically-constrained decoders (Post and Vilar, 2018; Hasler et al., 2018) …

Content truncated to 500 words