Morphological Inflection Generation with Hard Monotonic Attention

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 2004–2015 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1183 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 2004–2015 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1183 Morphological Inflection Generation with Hard Monotonic Attention Roee Aharoni & Yoav Goldberg Computer Science Department Bar-Ilan University Ramat-Gan, Israel {roee.aharoni,yoav.goldberg}@gmail.com Abstract We present a neural model for morpho- logical inflection generation which em- ploys a hard attention mechanism, inspired by the nearly-monotonic alignment com- monly found between the characters in a word and the characters in its inflec- tion. We evaluate the model on three pre- viously studied morphological inflection generation datasets and show that it pro- vides state of the art results in various se- tups compared to previous neural and non- neural approaches. Finally we present an analysis of the continuous representations learned by both the hard and soft atten- tion (Bahdanau et al., 2015) models for the task, shedding some light on the features such models extract. 1 Introduction Morphological inflection generation involves gen- erating a target word (e.g. “h¨artestem”, the German word for “hardest”), given a source word (e.g. “hart”, the German word for “hard”) and the morpho-syntactic attributes of the target (POS=adjective, gender=masculine, type=superlative, etc.). The task is important for many down-stream NLP tasks such as machine translation, especially for dealing with data sparsity in morphologically rich languages where a lemma can be inflected into many different word forms. Several studies have shown that translating into lemmas in the tar- get language and then applying inflection gener- ation as a post-processing step is beneficial for phrase-based machine translation (Minkov et al., 2007; Toutanova et al., 2008; Clifton and Sarkar, 2011; Fraser et al., 2012; Chahuneau et al., 2013) and more recently for neural machine translation (Garc´ıa-Mart´ınez et al., 2016). The task was traditionally tackled with hand en- gineered finite state transducers (FST) (Kosken- niemi, 1983; Kaplan and Kay, 1994) which rely on expert knowledge, or using trainable weighted finite state transducers (Mohri et al., 1997; Eisner, 2002) which combine expert knowledge with data- driven parameter tuning. Many other machine- learning based methods (Yarowsky and Wicen- towski, 2000; Dreyer and Eisner, 2011; Durrett and DeNero, 2013; Hulden et al., 2014; Ahlberg et al., 2015; Nicolai et al., 2015) were proposed for the task, although with specific assumptions about the set of possible processes that are needed to cre- ate the output sequence. More recently, the task was modeled as neu- ral sequence-to-sequence learning over character sequences with impressive results (Faruqui et al., 2016). The vanilla encoder-decoder models as used by Faruqui et al. compress the input sequence to a single, fixed-sized continuous representation. Instead, the soft-attention based sequence to se- quence learning paradigm (Bahdanau et al., 2015) allows directly conditioning on the entire input se- quence representation, and was utilized for mor- phological inflection generation with great success (Kann and …

Content truncated to 500 words