Modeling Word Formation in English–German Neural Machine Translation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4227–4232 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4227 Modeling Word Formation in English–German Neural Machine Translation Marion Weller-Di Marco and Alexander Fraser Center for Information and Language Processing LMU Munich {dimarco,fraser}@cis.uni-muenchen.de Abstract This paper studies strategies to model word formation in NMT using rich linguistic infor- mation, namely a word segmentation approach that goes beyond splitting into substrings by considering fusional morphology. Our linguis- tically sound segmentation is combined with a method for target-side inflection to accom- modate modeling word formation. The best system variants employ source-side morpho- logical analysis and model complex target-side words, improving over a standard system. 1 Introduction A major problem in word-level approaches to MT is a lack of morphological generalization. Both inflec- tional variants of the same lemma and derivations of a shared word stem are treated as unrelated. For morphologically complex languages with a large vocabulary, this is problematic, especially in low- resource or domain-adaption scenarios. A simple and widely used approach to reduce a large vocabulary in NMT is Byte Pair Encod- ing (BPE) (Sennrich et al., 2016), which itera- tively merges the top-frequent bigrams from ini- tially character-level split words until a set vocab- ulary size is reached. This strategy is effective, but linguistically uninformed and often leads to sub-optimal segmentation. Also, by only seg- menting words into substrings, BPE cannot handle non-concatenative operations, for example: • umlautung: BaumSg → B¨aumePl (tree/trees), • transitional elements that frequently occur in German compounds: Grenz|kontroll|politik → Grenze, Kontrolle (border control policy) • derivation: abundant ↔ abundance In this paper, we apply word segmentation on both the source and target sides that goes beyond merely splitting into exact substrings. This overcomes the issues caused by fusional morphology by accomo- dating modeling word formation across languages. Productive word formation can lead to a high number of infrequent word forms even though the morphemes in these words are frequent. A linguistically motivated segmentation method to handle processes such as compounding and derivation allows for better coverage and gen- eralization, both on the word level and on the morpheme level, and also enables the generation of new words. Sound morphological processing on the source and target side aims at learning productive word formation processes during translation, such as the English-German translation pair ungovernability↔Unregierbarkeit: un|PREF govern|V able|SUFF-ADJ ity|SUFF-NOUN ↔ un|PREF regieren|V bar|SUFF-ADJ keit|SUFF-NOUN Morphological information should not only handle isomorphic translation equivalents as above, but also help to uncover relations between source and target side for structurally different translations. 2 Related work There is a growing interest in the integration of lin- guistic information in NMT. For example, Eriguchi et al. (2016) and Bastings et al. (2017) demon- strate the positive impact of source-side syntactic information; N˘adejde et al. (2017) report improved translation quality when using syntactic informa- tion in form of CCG tags on source and target side. To address data sparsity, compound modeling has already been proved useful for phrase-based …

Content truncated to 500 words