Neural Morphological Tagging of Lemma Sequences for Machine Translation

Neural Morphological Tagging of Lemma Sequences for Machine Translation Costanza Conforti cc918@cam.ac.uk Matthias Huck mhuck@cis.lmu.de Alexander Fraser fraser@cis.lmu.de Center for Information and Language Processing, LMU Munich, Munich, Germany Abstract Translation to morphologically rich languages is a difficult task because of sparsity caused by morphological richness. In this work we perform a pilot study on predicting the morphologi- cally rich POS tags of sequences of lemmas. Similar studies have been conducted in the context of phrase-based statistical machine translation. We implement a state-of-the-art tagger taking lemmas as input and show that we can successfully predict the morphologically rich POS tags, with accuracies of up to 91%. 1 Introduction Modeling sequences of tokens in morphologically rich languages (MRLs) is a difficult task of great importance in many applications of NLP. For instance, translation from a morphologically poor language (such as English) to an MRL (such as German or Czech) is known to be difficult. An effective approach for modeling MRLs is to break the sequence into a factorized represen- tation, such as lemmas paired with their morphologically rich POS representations (e.g., for a German noun, the rich representation includes the noun POS tag, and the three grammatical features gender, number, and case). In this paper, we assume that we have a good system for generating lemmas and study whether we can automatically recover the morphologically rich POS representation. This is more difficult than morphologically rich POS tagging, which takes a sequence of surface forms and recovers the most likely morphologically rich POS representation, because lemma input is underspecified. This task was previously studied by Minkov et al. (2007). We differ in two ways: (1.) we implement a state-of-the-art neural tagger, rather than a Maximum Entropy Markov model, and (2.) we predict rich morphological POS, rather than surface forms. Studying the prediction of morphologically rich POS given lemmas is an interesting prob- lem in its own right. It has implications for NLP applications involving the generation of MRL sentences including machine translation. A concrete application is to apply it in an end-to-end MT system. Similar morphological prediction systems have been applied by Toutanova et al. (2008), Bojar and Kos (2010) and Fraser et al. (2012) in phrase-based SMT. A pipeline of such a system is depicted in Figure 1. Given the promising results in this initial study, we plan to combine our tagger with a standard neural machine translation model, resulting in a multi-task system which produces pairs of lemmas and morphologically rich POS tags. An important benefit of such a system over previous approaches which produce such pairs directly using a standard NMT model (e.g., Tamchyna et al. (2017)) is that we will be able to train it in a multi-task fashion, where some Proceedings of AMTA 2018, vol. 1: MT Research Track Boston, March 17 - 21, 2018 | Page 39 Figure 1: Example of a machine translation system using our morphology tagger (in red) as an intermediate step in the translation process into a target MRL. training examples contain source …

Content truncated to 500 words