Context Models for OOV Word Translation in Low-Resource Languages

Context Models for OOV Word Translation in Low-Resource Languages Angli Liu anglil@cs.washington.edu Department of Computer Science, University of Washington, Seattle, WA, 98195 Katrin Kirchhoff kk2@u.washington.edu Department of Electrical Engineering, University of Washington, Seattle, WA, 98195 Abstract Out-of-vocabulary word translation is a major problem for the translation of low-resource lan- guages that suffer from a lack of parallel training data. This paper evaluates the contributions of target-language context models towards the translation of OOV words, specifically in those cases where OOV translations are derived from external knowledge sources, such as dictio- naries. We develop both neural and non-neural context models and evaluate them within both phrase-based and self-attention based neural machine translation systems. Our results show that neural language models that integrate additional context beyond the current sentence are the most effective in disambiguating possible OOV word translations. We present an efficient second-pass lattice-rescoring method for wide-context neural language models and demon- strate performance improvements over state-of-the-art self-attention based neural MT systems in five out of six low-resource language pairs. 1 Introduction Translation of out-of-vocabulary (OOV) words (words occurring in the test data but not in the training data) is of major importance in statistical machine translation (MT). It is a particularly difficult problem in low-resource languages, i.e., languages for which parallel training data is extremely sparse, requiring recourse to techniques that are complementary to standard statisti- cal machine translation approaches. The approaches described in this paper were developed for scenarios where the training data comprises at most 100k sentences pairs. Most previous stud- ies in this area have focused on how to generate translation candidates for OOV words, either by segmentation into subword units, projection from other languages, or by leveraging external knowledge sources like dictionaries. Often, however, these methods generate multiple candi- dates for each OOV word, and the MT system is insufficiently trained to choose the appropriate translation according to the context. In this paper we address this problem by utilizing more sophisticated context models based on target-language information. In particular, we develop wide-context models that incorporate information from context beyond current sentence boundaries to resolve translation ambiguity. Proceedings of AMTA 2018, vol. 1: MT Research Track Boston, March 17 - 21, 2018 | Page 54 We compare these against models incorporating information from the current sentence only, and evaluate neural models vs. count-based sentence completion and graph reranking models. All are evaluated within both phrase-based and attention-based neural machine translation models for 6 low-resource language pairs. Our paper makes several contributions: • We evaluate recently proposed neural machine translation (NMT) architectures (purely attention-based neural MT) on low-resource languages and show that, contrary to previ- ous results obtained with sequence-to-sequence models, neural MT performs similarly to phrase-based machine translation (PBMT) in these scenarios, without modifications to the basic model. • We develop and compare several wide-context target-language based models for transla- tion disambiguation and find that document-level neural language models are the most effective at resolving translation ambiguities. • We present an efficient lattice rescoring algorithm for wide-context …

Content truncated to 500 words