A Language-Independent Unsupervised Model for Morphological Segmentation

Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 920–927, Prague, Czech Republic, June 2007. c⃝2007 Association for Computational Linguistics A Language-Independent Unsupervised Model for Morphological Segmentation Vera Demberg School of Informatics University of Edinburgh Edinburgh, EH8 9LW, GB v.demberg@sms.ed.ac.uk Abstract Morphological segmentation has been shown to be beneficial to a range of NLP tasks such as machine translation, speech recognition, speech synthesis and infor- mation retrieval. Recently, a number of approaches to unsupervised morphological segmentation have been proposed. This paper describes an algorithm that draws from previous approaches and combines them into a simple model for morpholog- ical segmentation that outperforms other approaches on English and German, and also yields good results on agglutinative languages such as Finnish and Turkish. We also propose a method for detecting variation within stems in an unsupervised fashion. The segmentation quality reached with the new algorithm is good enough to improve grapheme-to-phoneme conversion. 1 Introduction Morphological segmentation has been shown to be beneficial to a number of NLP tasks such as ma- chine translation (Goldwater and McClosky, 2005), speech recognition (Kurimo et al., 2006), informa- tion retrieval (Monz and de Rijke, 2002) and ques- tion answering. Segmenting a word into meaning- bearing units is particularly interesting for morpho- logically complex languages where words can be composed of several morphemes through inflection, derivation and composition. Data sparseness for such languages can be significantly decreased when words are decomposed morphologically. There ex- ist a number of rule-based morphological segmen- tation systems for a range of languages. However, expert knowledge and labour are expensive, and the analyzers must be updated on a regular basis in or- der to cope with language change (the emergence of new words and their inflections). One might argue that unsupervised algorithms are not an interesting option from the engineering point of view, because rule-based systems usually lead to better results. However, segmentations from an unsupervised algo- rithm that is language-independent are “cheap”, be- cause the only resource needed is unannotated text. If such an unsupervised system reaches a perfor- mance level that is good enough to help another task, it can constitute an attractive additional component. Recently, a number of approaches to unsupervised morphological segmentation have been proposed. These algorithms autonomously discover morpheme segmentations in unannotated text corpora. Here we describe a modification of one such unsupervised al- gorithm, RePortS (Keshava and Pitler, 2006). The RePortS algorithm performed best on English in a recent competition on unsupervised morphological segmentation (Kurimo et al., 2006), but had very low recall on morphologically more complex languages like German, Finnish or Turkish. We add a new step designed to achieve higher recall on morpho- logically complex languages and propose a method for identifying related stems that underwent regular non-concatenative morphological processes such as umlauting or ablauting, as well as morphological al- ternations along morpheme boundaries. The paper is structured as follows: Section 920 2 discusses the relationship between language- dependency and the level of supervision of a learn- ing algorithm. We …

Content truncated to 500 words