Neural Morphological Analysis: Encoding-Decoding Canonical Segments

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 961–967, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Neural Morphological Analysis: Encoding-Decoding Canonical Segments Katharina Kann Center for Information and Language Processing LMU Munich, Germany kann@cis.lmu.de Ryan Cotterell Department of Computer Science Johns Hopkins University, USA ryan.cotterell@jhu.edu Hinrich Sch¨utze Center for Information and Language Processing LMU Munich, Germany inquiries@cislmu.org Abstract Canonical morphological segmentation aims to divide words into a sequence of stan- dardized segments. In this work, we propose a character-based neural encoder- decoder model for this task. Additionally, we extend our model to include morpheme- level and lexical information through a neural reranker. We set the new state of the art for the task improving previous results by up to 21% accuracy. Our experiments cover three languages: English, German and Indonesian. 1 Introduction Morphological segmentation aims to divide words into morphemes, meaning-bearing sub-word units. Indeed, segmentations have found use in a diverse set of NLP applications, e.g., automatic speech recognition (Afify et al., 2006), keyword spot- ting (Narasimhan et al., 2014), machine transla- tion (Clifton and Sarkar, 2011) and parsing (Seeker and C¸ etino˘glu, 2015). In the literature, most re- search has traditionally focused on surface segmen- tation, whereby a word w is segmented into a se- quence of substrings whose concatenation is the en- tire word; see Ruokolainen et al. (2016) for a sur- vey. In contrast, we consider canonical segmenta- tion: w is divided into a sequence of standardized segments. To make the difference concrete, con- sider the following example: the surface segmen- tation of the complex English word achievability is achiev+abil+ity, whereas its canonical segmenta- tion is achieve+able+ity, i.e., we restore the alter- ations made during word formation. Canonical versions of morphological segmenta- tion have been introduced multiple times in the lit- erature (Kay, 1977; Naradowsky and Goldwater, 2009; Cotterell et al., 2016). Canonical segmen- tation has several representational advantages over surface segmentation, e.g., whether two words share a morpheme is no longer obfuscated by orthogra- phy. However, it also introduces a hard algorith- mic challenge: in addition to segmenting a word, we must reverse orthographic changes, e.g., map- ping achievability�→achieveableity. Computationally, canonical segmentation can be seen as a sequence-to-sequence problem: we must map a word form to a canonicalized version with segmentation boundaries. Inspired by the re- cent success of neural encoder-decoder models (Sutskever et al., 2014) for sequence-to-sequence problems in NLP, we design a neural architecture for the task. However, a na¨ıve application of the encoder-decoder model ignores much of the linguis- tic structure of canonical segmentation—it cannot directly model the individual canonical segments, e.g., it cannot easily produce segment-level embed- dings. To solve this, we use a neural reranker on top of the encoder-decoder, allowing us to embed both characters and entire segments. The combined approach outperforms the state of the art by a wide margin (up to 21% accuracy) in three languages: En- glish, German and Indonesian. 2 Neural Canonical Segmentation We begin by formally describing …

Content truncated to 500 words