Target-side Word Segmentation Strategies for Neural Machine Translation

Proceedings of the Conference on Machine Translation (WMT), Volume 1: Research Papers, pages 56–67 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics Target-side Word Segmentation Strategies for Neural Machine Translation Matthias Huck, Simon Riess, Alexander Fraser Center for Information and Language Processing LMU Munich Munich, Germany {mhuck,fraser}@cis.lmu.de,riess.simon@campus.lmu.de Abstract For efficiency considerations, state-of-the- art neural machine translation (NMT) re- quires the vocabulary to be restricted to a limited-size set of several thousand sym- bols. This is highly problematic when translating into inflected or compounding languages. A typical remedy is the use of subword units, where words are seg- mented into smaller components. Byte pair encoding, a purely corpus-based ap- proach, has proved effective recently. In this paper, we investigate word segmen- tation strategies that incorporate more lin- guistic knowledge. We demonstrate that linguistically informed target word seg- mentation is better suited for NMT, lead- ing to improved translation quality on the order of magnitude of +0.5 BLEU and −0.9 TER for a medium-scale English→German translation task. Our work is important in that it shows that linguistic knowledge can be used to im- prove NMT results over results based only on the language-agnostic byte pair encod- ing vocabulary reduction technique. 1 Introduction Inflection and nominal composition are morpho- logical processes which exist in many natural lan- guages. Machine translation into an inflected lan- guage or into a compounding language must be capable of generating words from a large vocabu- lary of valid word surface forms, or ideally even be open-vocabulary. In NMT, though, dealing with a very large number of target symbols is expensive in practice. While, for instance, a standard dictionary of German, a compounding language, may cover 140 000 vocabulary entries,1 NMT on off-the- shelf GPU hardware is nowadays typically only tractable with target vocabularies below 100 000 symbols. This issue is made worse by the fact that com- pound words are not a closed set. More frequently occurring compound words may be covered in a standard dictionary (e.g., “Finanztransaktions- steuer”, English: “financial transaction tax”), but the compounding process allows for words to be freely joined to form new ones (e.g., “Finanztrans- aktionssteuerzahler”, English: “financial transac- tion tax payer”), and compounding is highly pro- ductive in a language like German. Furthermore, a dictionary lists canonical word forms, many of which can have many inflected variants, with morphological variation depending on case, number, gender, tense, aspect, mood, and so on. The German language has four cases, three grammatical genders, and two numbers. Ger- man exhibits a rich amount of morphological word variations also in the verbal system. A machine translation system should ideally be able to pro- duce any permissible compound word, and all in- flections for each canonical form of all words (in- cluding compound words). Previous work has drawn on byte pair encod- ing to obtain a fixed-sized vocabulary of subword units. In this paper, we investigate word segmen- tation strategies for NMT which are linguistically more informed. Specifically, we explore and em- pirically compare: • Compound …

Content truncated to 500 words