Neural Sequence-to-sequence Learning of Internal Word Structure

Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 184–194, Vancouver, Canada, August 3 - August 4, 2017. c⃝2017 Association for Computational Linguistics Neural Sequence-to-sequence Learning of Internal Word Structure Tatyana Ruzsics Tanja Samardˇzi´c CorpusLab, URPP Language and Space, University of Zurich, Switzerland {tatiana.ruzsics, tanja.samardzic}@uzh.ch Abstract Learning internal word structure has re- cently been recognized as an important step in various multilingual processing tasks and in theoretical language com- parison. In this paper, we present a neural encoder-decoder model for learn- ing canonical morphological segmenta- tion. Our model combines character-level sequence-to-sequence transformation with a language model over canonical seg- ments. We obtain up to 4% improvement over a strong character-level encoder- decoder baseline for three languages. Our model outperforms the previous state-of- the-art for two languages, while eliminat- ing the need for external resources such as large dictionaries. Finally, by compar- ing the performance of encoder-decoder and classical statistical machine transla- tion systems trained with and without cor- pus counts, we show that including corpus counts is beneficial to both approaches. 1 Introduction One of the most obvious structural differences be- tween languages is the variation in the complex- ity of internal word structure. In some languages, such as English, words are relatively short and morphologically less complex. In other languages, such as Chintang in Example 11, words tend to be long and encapsulate rather rich structure. The Chintang verb thaptakha in Example 1 consists of a number of morphemes expressing the impera- tive mode, aspect and deixis. The information ex- pressed by a single Chintang verb requires several 1The example is adapted from (Stoll et al., In press) words in English, as it can be seen in the glosses.2 and in the translation. a.Example 1 cuwa thaptakha b. cuwa thapt -a -khag -a c. water move -IMP -see -IMP across [2sS] d. Bring some water over here! The variation in word structure is observed even in common categories such as plural, which is typ- ically part of a word, but expressed using different structures. The items in Example 23 show three different structural types associated with express- ing plural across languages. Example 2 Type Sg. Pl. Suffix Turkish ev ev-ler ‘house’ ‘houses’ Prefix Swahili m-toto wa-toto ‘child’ ‘children’ Redupli- Malay anak anak-anak cation ‘child’ ‘children’ With the spread of natural language process- ing to a wider range of languages, learning in- ternal word structure becomes increasingly impor- tant for developing practical applications. Analy- sis of internal word structure, usually termed mor- phological segmentation, has been shown helpful in tasks such as machine translation (Dyer et al., 2008; Narasimhan et al., 2014), speech process- ing (Creutz et al., 2007) and parsing (Seeker and C¸ etinoglu, 2015). Additionally, there is a grow- ing interest in automatic learning of morphologi- cal segmentation for the purpose of theoretical lan- 2Cf. Leipzig Glossing Rules at https://www.eva. mpg.de/lingua/resources/glossing-rules. php 3The examples are adapted from (Eifring and Theil, 2005) 184 guage comparison (Bentz et al., 2017). In this con- text, it becomes …

Content truncated to 500 words