HABLex: Human Annotated Bilingual Lexicons for Experiments in Machine Translation

1382 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1382–1387, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics HABLex: Human Annotated Bilingual Lexicons for Experiments in Machine Translation Brian Thompson∗ Rebecca Knowles∗ Xuan Zhang∗ Huda Khayrallah Kevin Duh Philipp Koehn Johns Hopkins University {brian.thompson, rknowles, xuanzhang, huda, phi}@jhu.edu kevinduh@cs.jhu.edu Abstract Bilingual lexicons are valuable resources used by professional human translators. While these resources can be easily incorporated in statistical machine translation, it is un- clear how to best do so in the neural frame- work. In this work, we present the HABLex dataset, designed to test methods for bilin- gual lexicon integration into neural machine translation. Our data consists of human- generated alignments of words and phrases in machine translation test sets in three language pairs (Russian→English, Chinese→English, and Korean→English), resulting in clean bilin- gual lexicons which are well matched to the reference. We also present two simple baselines—constrained decoding and contin- ued training—and an improvement to contin- ued training to address overfitting. 1 Introduction Neural machine translation (NMT) is the current state-of-the-art. In contrast with statistical ma- chine translation (SMT; Koehn et al., 2007), where there are several established methods of incorpo- rating external knowledge,1 recent work is still ex- amining how best to incorporate bilingual lexicons into NMT systems. Bilingual lexicon integration is desirable in a number of scenarios: highly tech- nical vocabulary (which might be rare, or require translations of a domain-specific sense), lower- resource settings (where bilingual lexicons might be a significant portion of the available parallel data), translation settings where a client requires particular terms to be used (e.g. brand names), or for improving rare word translation. At present, there is no standard dataset for benchmarking bilingual lexicon integration, mak- ing it difficult to compare methods. ∗These authors contributed equally to this work. 1E.g. requiring a particular translation via XML markup: statmt.org/moses/?n=advanced.hybrid Source на передней стенке корпуса уста- новлены амортизаторы . Lexical Entry амортизаторы↔shock absorbers Target shock absorbers are mounted on a front wall of the housing . Table 1: Example Russian source sentence, lexicon en- try, and English target sentence. We create and release2 Human Annotated Bilingual Lexicons (HABLex). Our bilingual lex- icons are (1) generated by bilingual experts to en- sure high quality, (2) derived from the develop- ment and test references so the best-case-scenario impact on translation performance can be directly measured, (3) covering 3 language pairs, and (4) focused on challenging words. We perform exploratory work on our develop- ment set, showing two representative baselines to compare incorporating the lexicon at training time (continued training) vs. at decoding time (con- strained decoding). We examine the tradeoffs in terms of BLEU, recall, and speed. We also present a novel application of Elastic Weight Consolidation (EWC; Kirkpatrick et al., 2017; Thompson et al., 2019) which significantly improves performance by preventing overfitting during continued training on the bilingual lexicon. 2 Related Work We first review prior …

Content truncated to 500 words